<< 06-02-2025 >>
01:35:48FromDiscord<arkanoid> I wow, there is a cycle detector that can be used to track if code is arc candidate\: https://github.com/nim-lang/Nim/pull/23272
03:25:21*rockcavera quit (Remote host closed the connection)
06:10:29FromDiscord<yummy_licorice> is it possible to get other values from -d:
06:10:57FromDiscord<yummy_licorice> like something like -d:libPath:/usr/include/somelib
06:17:48FromDiscord<bostonboston> strdefine
06:21:31FromDiscord<yummy_licorice> thanks
06:30:28*alexdaguy joined #nim
07:46:09*redj quit (Ping timeout: 276 seconds)
07:46:28*redj joined #nim
07:52:56*derpydoo joined #nim
07:55:27FromDiscord<luteva> sent a long message, see https://pasty.ee/kvZLZOFq
07:59:11FromDiscord<Elegantbeef> What compile time errors are you event trying to test for?
08:10:57*jjido joined #nim
08:42:04*tokyovigilante quit (Ping timeout: 260 seconds)
09:14:58*tokyovigilante joined #nim
09:16:16FromDiscord<nocturn9x> sent a code paste, see https://play.nim-lang.org/#pasty=AjTOiVwH
09:16:35FromDiscord<nocturn9x> sent a code paste, see https://play.nim-lang.org/#pasty=QLmcQPpT
09:16:48FromDiscord<nocturn9x> like wtf you mean I gave you the exact fucking thing you want???
09:19:29FromDiscord<nocturn9x> interestingly I get an error in vscode on the line `ft: ...` saying "Layer[...] is not a concrete type", but the nim compiler doesn't seem to catch it
09:19:37FromDiscord<nocturn9x> am I fundamentally misunderstanding something
09:21:01*tokyovigilante quit (Ping timeout: 268 seconds)
09:23:10Amun-Rathat it's not the compiler that's analyzing the code
09:25:12*tokyovigilante joined #nim
09:34:41FromDiscord<nocturn9x> yeah
09:34:52FromDiscord<nocturn9x> I'm still confused as to why it doesn't compile though lol
09:46:24*jjido quit (Quit: My laptop has gone to sleep. ZZZzzz…)
09:48:40FromDiscord<nocturn9x> playground snippet: https://play.nim-lang.org/#pasty=slrVqkmp
09:58:17*jjido joined #nim
09:59:48FromDiscord<luteva> In reply to @Elegantbeef "What compile time errors": e.g. type errors. just what z
09:59:52FromDiscord<lainlaylie> smaller reproduction: https://wandbox.org/permlink/nubKyW87vflvKXna
10:00:16FromDiscord<lainlaylie> the usual `distinct` trick from implicit generic procs doesn't seem to work here
10:00:29FromDiscord<luteva> (edit) "z" => "static code analyzis does (which uses the compiler/VM/nimsuggest)"
10:01:25FromDiscord<luteva> (edit) "compiler/VM/nimsuggest)" => "compiler/VM/nimsuggest)↵Oh! You mean: In the tests? Well I am talking about unit test e.g."
10:05:06*jjido quit (Quit: My laptop has gone to sleep. ZZZzzz…)
10:36:58FromDiscord<Robyn [She/Her]> In reply to @nocturn9x "can someone enlighten me": I think it's because you're using type aliases and you have `FTWeightBias` as well as `IntWeight` both being int16
10:37:08FromDiscord<Robyn [She/Her]> Either use different types or try a distinct type
10:37:39FromDiscord<nocturn9x> huh
10:39:39FromDiscord<nocturn9x> sent a code paste, see https://play.nim-lang.org/#pasty=CrIxRagC
10:39:42FromDiscord<nocturn9x> this still does not compile
10:39:48FromDiscord<nocturn9x> `/usercode/in.nim(27, 9) Error: 'Layer[3072, 256, system.int16, system.int16]' is not a concrete type`
10:39:55FromDiscord<nocturn9x> that would be `l1`
10:52:37FromDiscord<nocturn9x> anyway I have another question
10:52:51FromDiscord<nocturn9x> I want to read a file from disk at compile time containing a struct and cast it to the correct type
10:52:57FromDiscord<nocturn9x> it appears Nim's VM does not support this
10:53:16FromDiscord<nocturn9x> how can I get around it? I need this to be a constant, or it won't be put in the static segment of the executable
10:54:27FromDiscord<nocturn9x> `const DEFAULT_NET_VERBATIM = cast[Network](staticRead("../verbatim.bin"))`
10:54:38FromDiscord<nocturn9x> this errors out with `VM does not support 'cast' from tyString to tyObject`
10:55:51FromDiscord<nocturn9x> ig I could put the cast in the runtime loading part where I just assign the variable
11:09:10Amun-Radon't use cast, initialize object nim-way
11:30:56FromDiscord<nocturn9x> why would it not work
11:31:12FromDiscord<nocturn9x> I'm doing `file.write(cast[ptr UncheckedArray[byte]](addr net))`
11:31:23FromDiscord<nocturn9x> but when loading it back in it fails
11:31:34FromDiscord<nocturn9x> I can't initialize the object the nim way lmao I just have a stream of bytes
11:32:06FromDiscord<nocturn9x> it seems like something is getting corrupted
11:32:28FromDiscord<nocturn9x> and no, I don't need `{.packed.}` since everything is already aligned to 64 bytes and I want to preserve that padding in the final result
11:34:26Amun-Rayou can always initialize an object
11:34:54FromDiscord<nocturn9x> wut
11:34:58FromDiscord<nocturn9x> it's the result of a procedure
11:35:02FromDiscord<nocturn9x> it's already going to be initialized
11:35:03FromDiscord<nocturn9x> lol
11:35:34Amun-Rayou can initialize a Network object using initNetwork equivalent: const DEFAULT_NET_VERBATIM = cast[Network](staticRead("../verbatim.bin"))`
11:35:52Amun-Raand file.write + UncheckedArray makes no sense
11:36:04Amun-Rathere's no size information
11:36:10FromDiscord<nocturn9x> In reply to @Amun-Ra "you can initialize a": that is literally what I have tried
11:36:16FromDiscord<nocturn9x> In reply to @Amun-Ra "and file.write + UncheckedArray": yeah I modified it
11:36:22FromDiscord<nocturn9x> In reply to @nocturn9x "that is literally what": teh VM does not support this lmao
11:36:32Amun-Rasupport what?
11:36:33FromDiscord<nocturn9x> `file.writeData(cast[ptr UncheckedArray[byte]](addr net), sizeof(Network))`
11:36:35FromDiscord<nocturn9x> this is the line
11:36:44FromDiscord<nocturn9x> In reply to @Amun-Ra "support what?": it says it doesn't support the cast operation
11:36:45FromDiscord<nocturn9x> 🤷
11:36:53Amun-Rathat is what I'm saying
11:37:16FromDiscord<nocturn9x> how do I say this
11:37:19Amun-Rado you have that function somewhere online?
11:37:27FromDiscord<nocturn9x> why do you think I would be bothering with this
11:37:32FromDiscord<nocturn9x> if I could just initialize the network normally
11:37:37FromDiscord<nocturn9x> the point is to not have copies.
11:37:50FromDiscord<nocturn9x> initializing the network would require copying the weights to the object
11:37:54FromDiscord<nocturn9x> which is what I am already doing
11:37:57FromDiscord<nocturn9x> and is not what I want
11:38:00Amun-Raare we still talking compile time?
11:38:06FromDiscord<nocturn9x> what I want is for the network to be in the .static segment of the executable
11:38:22FromDiscord<nocturn9x> such that if more instances of my engine are run on the same machine, they'll all use the same network data
11:38:24FromDiscord<nocturn9x> without overhead
11:38:32FromDiscord<nocturn9x> since the executable is loaded in memory only once
11:38:39FromDiscord<nocturn9x> and they all access the same read only section of the binary
11:38:55FromDiscord<nocturn9x> any sort of runtime manipulation breaks that behavior
11:39:05FromDiscord<nocturn9x> so I can't initialize the object in any way, I need casting to work
11:39:07FromDiscord<nocturn9x> that's it
11:39:22Amun-Ratbh there wouldn't be much overhead if those objects were duplicated, apart from memory overhead
11:39:26FromDiscord<nocturn9x> sent a code paste, see https://play.nim-lang.org/#pasty=UnyKpRyc
11:39:34FromDiscord<nocturn9x> In reply to @Amun-Ra "tbh there wouldn't be": the memory overhead is literally the problem
11:39:48FromDiscord<nocturn9x> believe me, I know what I'm doing. This is standard practice for advanced chess engines.
11:40:05FromDiscord<nocturn9x> (edit) "https://play.nim-lang.org/#pasty=WMiQbFTz" => "https://play.nim-lang.org/#pasty=puMKxrkt"
11:40:13FromDiscord<nocturn9x> sent a code paste, see https://play.nim-lang.org/#pasty=uyBuysCv
11:40:22FromDiscord<nocturn9x> what I need is for typecasting to work
11:40:34FromDiscord<nocturn9x> but I'm likely doing something wrong idk
11:40:45FromDiscord<nnsee> In reply to @nocturn9x "the memory overhead is": surely this is what references are for
11:41:04FromDiscord<nocturn9x> In reply to @nnsee "surely this is what": ah yes the famous references that cross process boundaries
11:41:05FromDiscord<nocturn9x> ????
11:41:14FromDiscord<nnsee> .static doesn't cross process boundaries either
11:41:19Amun-Ratbh I don't see a problem in replacing cast with init, keeping object unique
11:41:19FromDiscord<nocturn9x> it does
11:41:21FromDiscord<nnsee> every process gets allocated its own .static
11:41:34FromDiscord<nocturn9x> well then every top engine developer is a moron I guess
11:41:48FromDiscord<nocturn9x> cuz it works fine in C++ and gains measurable amounts of elo when running multiple instances on the same machine
11:41:55FromDiscord<nocturn9x> the runtime equivalent of this would be to mmap + cast
11:42:02Amun-Raso you're saying every one of them tried to cast in compile-time nim code
11:42:15FromDiscord<nocturn9x> I'm saying they've done it in C++ without issue
11:42:32FromDiscord<nocturn9x> the casting isn't even the problem, I can just have that outside of the constant
11:42:34FromDiscord<nocturn9x> it has no runtime cost
11:42:41FromDiscord<nocturn9x> the problem is I can't get the object to dump to disk correctly
11:43:33FromDiscord<nocturn9x> so the question is how do I dump an arbitrary object to a binary file and then load it back in as a string using staticRead such that at "runtime" I can do `net = cast[Network](data)`?
11:43:45FromDiscord<nnsee> In reply to @nocturn9x "well then every top": interesting. apparently i'm wrong
11:44:01FromDiscord<nnsee> ie most OSs do the same physical pages but apply some sort of CoW for it
11:44:05Amun-Ramake a proc that parses a string returned from staticRead
11:44:07FromDiscord<nocturn9x> yes
11:44:09FromDiscord<nnsee> didn't know this, TIL
11:44:10FromDiscord<nocturn9x> that is the usual way
11:44:32FromDiscord<nocturn9x> In reply to @Amun-Ra "make a proc that": hmm that could work but it would make compile times slow as fuck since it'd be iterating ~18 million weights
11:44:38FromDiscord<nocturn9x> I tested this before
11:44:52FromDiscord<nocturn9x> In reply to @nnsee "ie most OSs do": and since the network weights are never written to
11:44:55FromDiscord<nocturn9x> (by design)
11:44:57FromDiscord<nocturn9x> then you get a free copy
11:45:00FromDiscord<nocturn9x> with no overhead
11:45:02FromDiscord<nocturn9x> it's pretty nifty
11:45:10Amun-Rayou haven't mentioned that before, convert once and keep the result in cache
11:45:23FromDiscord<nocturn9x> huh how do I do that
11:46:23FromDiscord<nnsee> In reply to @nocturn9x "ah yes the famous": in any case i assumed you were talking about threads and not processes
11:46:35FromDiscord<nocturn9x> also remind me what was the pure nim way of swapping two bytes?
11:46:36FromDiscord<nnsee> but yes, what you're trying to do makes sense
11:46:46FromDiscord<nocturn9x> In reply to @nnsee "in any case i": well that wouldn't be a problem
11:46:53FromDiscord<nocturn9x> the network weights are already shared across threads
11:46:56FromDiscord<nocturn9x> since it's a global
11:49:16FromDiscord<nnsee> if you were targeting just unix i would just recommend mmap with MAP_SHARED, but i'm guessing you want your engine to work on windows as well
11:50:16FromDiscord<nocturn9x> yeah
11:50:26FromDiscord<nocturn9x> some engines bother with that
11:50:36FromDiscord<nocturn9x> they decompress the net in a shared memory segment under /tmp and mmap that
11:50:37FromDiscord<nocturn9x> but ehh
11:50:56FromDiscord<nocturn9x> windows is an important target for my engine unfortunately
11:50:59FromDiscord<nocturn9x> many testers use it
11:52:38FromDiscord<nnsee> wait, remind me again why casting to the object at runtime doesn't work?
11:53:31FromDiscord<nocturn9x> idk it gets fucked
11:53:37FromDiscord<nocturn9x> my guess is the write part is the problem
11:53:44FromDiscord<nocturn9x> but it is just not reconstructed correctly
11:54:31FromDiscord<nocturn9x> I will try a different approach where instead of embedding the network weights in the binary and then load those into the network object at runtime, I just read those at compile time and load them directly into a default network object
11:55:36FromDiscord<nocturn9x> (edit) "I will try a different approach where instead of embedding the network weights in the binary and then load those into the network object at runtime, I just read those at compile time and load them directly into a default network object ... " added "still at compile time"
11:56:19FromDiscord<nocturn9x> ah and of course
11:56:22FromDiscord<nocturn9x> sent a code paste, see https://play.nim-lang.org/#pasty=jZCSzHaK
11:56:25FromDiscord<nocturn9x> same error as yesterday
11:56:30FromDiscord<nocturn9x> lmao the VM is so fucked
11:59:37FromDiscord<nocturn9x> hysterical
11:59:47FromDiscord<nnsee> In reply to @nocturn9x "idk it gets fucked": can it be that you're casting a string type (returned by the staticRead) which has the length and the reserved field prepended to it, to your object which probably doesn't have those. i haven't actually done nim in a while so i forget if this is how it works
12:00:10FromDiscord<nocturn9x> I am doing exactly that
12:00:19FromDiscord<nocturn9x> upsie?
12:00:39FromDiscord<nocturn9x> can I like convert to cstring or something
12:00:47FromDiscord<nocturn9x> that would be null terminated tho
12:00:47FromDiscord<nocturn9x> hm
12:00:55FromDiscord<nocturn9x> I just need a raw stream of bytes 😭
12:03:22FromDiscord<nnsee> add the same fields to your object and just treat it as arbitrary padding?
12:03:56FromDiscord<nnsee> and when writing it, cut those bytes from the beginning? idk how you're creating the binary data
12:04:00FromDiscord<nocturn9x> where can I find the string definition
12:04:10FromDiscord<nocturn9x> In reply to @nnsee "and when writing it,": literally just reading off from staticRead
12:04:11FromDiscord<nocturn9x> lol
12:04:15FromDiscord<nnsee> i'm currently looking for it
12:04:31FromDiscord<nocturn9x> `const DEFAULT_NET = when INCBIN: cstring(staticRead(DEFAULT_NET_PATH)) else: Network()`
12:04:38FromDiscord<nocturn9x> and then
12:04:46FromDiscord<nocturn9x> sent a code paste, see https://play.nim-lang.org/#pasty=CooieCvn
12:04:58FromDiscord<nocturn9x> this currently does not work
12:05:15FromDiscord<nocturn9x> ahh ofc it can't work cuz of the padding
12:05:16FromDiscord<nocturn9x> nvm
12:05:17FromDiscord<nocturn9x> I'm stupid
12:05:22FromDiscord<nocturn9x> I need to dump the padded net to disk
12:05:27FromDiscord<nocturn9x> ugh this is so remarkably annoying fr
12:05:59FromDiscord<nocturn9x> this is how I dump it btw
12:06:00FromDiscord<nocturn9x> `file.writeData(cast[ptr UncheckedArray[byte]](addr net), sizeof(Network))`
12:07:48FromDiscord<nocturn9x> sent a code paste, see https://play.nim-lang.org/#pasty=OdvuLEMV
12:07:51FromDiscord<nocturn9x> and this is the object itself
12:10:27*ftajhii quit (Ping timeout: 246 seconds)
12:11:04FromDiscord<nocturn9x> anyway I've dumped the raw network object to disk with that line of code and tried loading back in via a cast
12:11:06FromDiscord<nocturn9x> and it no workie
12:11:06FromDiscord<nocturn9x> so idk
12:11:14FromDiscord<nocturn9x> not even converting to cstring
12:36:40*derpydoo quit (Quit: derpydoo)
12:46:56FromDiscord<narimiran> Nim 2.2.2 is out!! ↵↵https://nim-lang.org/blog/2025/02/05/nim-222.html
13:02:37FromDiscord<enthus1ast.> In reply to @nocturn9x "so the question is": I would use https://github.com/treeform/flatty for this, idk if it would work on compile time, though.
13:03:33FromDiscord<enthus1ast.> there would also be no need for casting
13:03:56FromDiscord<nocturn9x> that looks nice
13:03:57FromDiscord<nocturn9x> hmm
13:13:29FromDiscord<nocturn9x> quick question about threading
13:13:48FromDiscord<nocturn9x> what is the most painless way you know of migrating from the old `createThread` paradigm to a thread pool?
13:13:59FromDiscord<nocturn9x> I've tried to do that many times and always failed
13:14:43FromDiscord<nocturn9x> not modifying my code would be nice, anything more than just replacing the `createThread` call with `something.submit()` is a no-go
13:14:48FromDiscord<nocturn9x> and I need shared heaps to work ofc
13:27:36FromDiscord<enthus1ast.> i personally still use createThread ...
13:29:13FromDiscord<enthus1ast.> but i know that some use https://github.com/Araq/malebolgia now
13:44:02FromDiscord<mratsim> In reply to @nocturn9x "not modifying my code": submit is primitive on top of createThread if people don't want to manage their threadpools / event loops
13:44:20FromDiscord<nocturn9x> In reply to @enthus1ast. "but i know that": that works surprisingly poorly
13:44:21FromDiscord<nocturn9x> tried it before
13:44:35FromDiscord<nocturn9x> In reply to @mratsim "submit is primitive on": can I get an example of how that works?
13:45:15FromDiscord<mratsim> In reply to @nocturn9x "can I get an": https://github.com/mratsim/weave-io/blob/master/weave_io/threadpool.nim#L940-L955
13:45:34FromDiscord<mratsim> all threadpools, malebolgia, weave, taskpool, follow that pattern.
13:46:29FromDiscord<nocturn9x> I need something a bit higher level
13:46:40FromDiscord<nocturn9x> where I just instantiate a thread pool of a given size and submit procedures to it
13:47:23FromDiscord<mratsim> but you said that `something.submit()` is a no go?
13:47:43FromDiscord<nocturn9x> nono
13:47:55FromDiscord<nocturn9x> what I said is that anything more complex than `something.submit` is a no-go
13:47:59FromDiscord<mratsim> ah
13:48:21FromDiscord<mratsim> https://github.com/status-im/nim-taskpools
13:48:28FromDiscord<mratsim> It only does that.
13:48:51FromDiscord<nocturn9x> <https://git.nocturn9x.space/heimdall-engine/heimdall/src/branch/master/src/heimdall/search.nim#L1397-L1435>
13:48:54FromDiscord<nocturn9x> this is how I currently do it
13:49:02FromDiscord<mratsim> The api is intentionally simple so it's easy to audit: https://github.com/status-im/nim-taskpools?tab=readme-ov-file#api
13:49:44FromDiscord<nocturn9x> I think I tried nim-taskpools before
13:49:48FromDiscord<nocturn9x> with fairly disastrous results
13:49:50FromDiscord<nocturn9x> let me test once again
13:49:54FromDiscord<nocturn9x> seems simple enough
13:50:46FromDiscord<nocturn9x> I don't even need to handle return values or anything of the sort
13:50:49FromDiscord<mratsim> you shouldn't createThread within a loop, it takes like 10ms to create a thread. You should create it outside and then each worker wait on a queue and process task that are enqueued there..
13:51:00FromDiscord<nocturn9x> I am aware this is suboptimal
13:51:08FromDiscord<nocturn9x> trying to refactor it has failed
13:51:12FromDiscord<nocturn9x> (edit) "trying to refactor it has failed ... " added "many times"
13:51:15FromDiscord<enthus1ast.> In reply to @mratsim "you shouldn't createThread within": this is how i use it
13:51:30FromDiscord<nocturn9x> I assume taskpools does this
13:52:04FromDiscord<mratsim> sent a long message, see https://pasty.ee/aEIPTWyn
13:52:30FromDiscord<nocturn9x> on windows the C codegen for it is broken too
13:52:33FromDiscord<nocturn9x> I'll get rid of it
13:52:38FromDiscord<mratsim> In reply to @nocturn9x "I assume taskpools does": it does this but instead of having a single queue, each worker has its queue and when their queue is empy they steal from each other.
13:52:52FromDiscord<nocturn9x> oo
13:52:53FromDiscord<nocturn9x> fancy
13:53:35FromDiscord<mratsim> this removes contention from many threads hammering a single queue (which needs syunchronization)↵which is crucial for small tasks.
13:53:52FromDiscord<nocturn9x> my tasks are pretty chonktastic tbh
13:53:59FromDiscord<mratsim> it doesn't matter for a task longer than 100ms ~ 1s though
13:54:00FromDiscord<nocturn9x> the engine search is long running and expensive
13:54:07FromDiscord<nocturn9x> yeah figures
13:54:21FromDiscord<nocturn9x> like a search in a multithreaded environments can take minutes
13:54:25FromDiscord<nocturn9x> several
13:54:33FromDiscord<nocturn9x> (edit) "minutes" => "minute"
13:54:41FromDiscord<nocturn9x> (edit) "like a search in a ... multithreadedenvironment" added "highly" | "environments" => "environment" | "minute" => "minutes"
13:54:44FromDiscord<mratsim> but if you have a lot of for loop it can help distributing them efficiently if one thread is starving and the other has work
13:54:55FromDiscord<nocturn9x> yeah that makes sense
13:55:10FromDiscord<mratsim> especially on CPUs with P and E core that process stuff at very different speeds
13:56:33FromDiscord<nocturn9x> yep
13:58:42FromDiscord<nocturn9x> sent a code paste, see https://play.nim-lang.org/#pasty=nASuXDoL
13:58:47FromDiscord<nocturn9x> does this setup logic look fine?
13:59:01FromDiscord<nocturn9x> the number of threads can change at any time as long as a search is not in progress, so I need to handle this
13:59:11FromDiscord<nocturn9x> (it is set with a user facing option)
13:59:40FromDiscord<nocturn9x> ah ig I need to handle the case when it is one
13:59:47FromDiscord<nocturn9x> since one worker just means doing a single threaded search
13:59:51FromDiscord<nocturn9x> which is done in the main thread
14:00:02FromDiscord<nocturn9x> (the search is spawned in a separate thread via `createThread`)
14:01:28FromDiscord<nocturn9x> sent a code paste, see https://play.nim-lang.org/#pasty=NYZfqCCE
14:06:22FromDiscord<nocturn9x> hmm
14:06:25FromDiscord<nocturn9x> sent a code paste, see https://play.nim-lang.org/#pasty=bVDiXmzY
14:07:13FromDiscord<nocturn9x> sent a code paste, see https://play.nim-lang.org/#pasty=rWRUkzrV
14:07:38FromDiscord<nocturn9x> (edit) "https://play.nim-lang.org/#pasty=rjprgHLa" => "https://play.nim-lang.org/#pasty=AdBeUweO"
14:08:05FromDiscord<nocturn9x> how do I tell it to shut up? Putting `{.raises: [].}` just causes the problem to be diverted up the call tree
14:08:14FromDiscord<nocturn9x> and I don't really want to mark every function I call like that lol
14:09:31FromDiscord<nocturn9x> how do I tell it to just stfu
14:09:41FromDiscord<nocturn9x> all exceptions are turned to panics anyway, they're all fatal errors
14:11:51FromDiscord<xtrayambak> In reply to @nocturn9x "how do I tell": Is there any particular reason you're doing that? Violating GC safety isn't my goto solution for this stuff
14:12:05FromDiscord<xtrayambak> Unless a library I'm using just doesn't have a gcsafe function I'm using
14:12:07FromDiscord<nocturn9x> there is no other way to make it work
14:12:17FromDiscord<nocturn9x> none of the things I'm doing can ever be inferred to be gcsafe
14:12:32FromDiscord<xtrayambak> Yikes, I'm pretty sure you have to do what you fear then
14:12:49FromDiscord<nocturn9x> there's loads of sharing and globals and stuff that are logically safe since they're either read only or written to at very specific, known safe intervals
14:12:57FromDiscord<nocturn9x> (or in the case of the TT, collisions just don't matter)
14:13:06FromDiscord<nocturn9x> the compiler will never be able to catch that
14:13:17FromDiscord<nocturn9x> I mean I say loads, actually it's not that many anymore
14:13:21FromDiscord<nocturn9x> just in a few critical places
14:13:32FromDiscord<nocturn9x> but they are very far reaching
14:13:48FromDiscord<nocturn9x> like for example the attack tables for move generation are constant size arrays shared across threads
14:13:57FromDiscord<nocturn9x> they're only ever read from so it's a non-issue
14:14:02FromDiscord<nocturn9x> ig nim would be able to catch that
14:14:05FromDiscord<nocturn9x> but the rest is more nuanced
14:14:19FromDiscord<nocturn9x> and I do use atomics for many things just to be on the safe side
14:14:33FromDiscord<xtrayambak> I've never done any serious multithreading in Nim before, I used [weave](https://github.com/mratsim/weave) to offload image buffer drawing tasks to another thread in my [rendering engine](<https://github.com/ferus-web/ferusgfx>)
14:14:47FromDiscord<nocturn9x> sent a code paste, see https://play.nim-lang.org/#pasty=NxcKaiFz
14:14:48FromDiscord<nocturn9x> welp
14:14:49FromDiscord<nocturn9x> here we go
14:14:52FromDiscord<nocturn9x> time to debug
14:15:15FromDiscord<xtrayambak> What library are you using for threadpools? Weave, malbolgia or your own thing?
14:15:25FromDiscord<nocturn9x> nim-taskpools
14:15:30FromDiscord<xtrayambak> Ah
14:15:39FromDiscord<xtrayambak> Try weave, it works fairly well from my experience
14:16:11FromDiscord<nocturn9x> sent a code paste, see https://play.nim-lang.org/#pasty=jjpSrcPM
14:16:13FromDiscord<nocturn9x> I was being silly
14:16:14FromDiscord<nocturn9x> logic mistake
14:16:19FromDiscord<xtrayambak> In my case (the rendering engine), the thread never has to interact with any GC unsafe segments (like accessing global variables) so it just worked well with weave
14:16:47FromDiscord<xtrayambak> mratsim is just a genius to be honest :P
14:17:21FromDiscord<xtrayambak> In reply to @nocturn9x "logic mistake": Haha I remember trying to write a multithreaded HTTP server with no prior experience, I managed to make glibc throw errors at me
14:17:32FromDiscord<xtrayambak> (edit) "In reply to @nocturn9x "logic mistake": Haha I remember trying to write a multithreaded HTTP server with no prior experience, I managed to make glibc throw ... errors" added "internal"
14:17:58FromDiscord<nocturn9x> rofl
14:18:29FromDiscord<nocturn9x> sent a code paste, see https://play.nim-lang.org/#pasty=uHmxHlLI
14:18:38FromDiscord<nocturn9x> cool so it appears something is not working as intended
14:20:52FromDiscord<xtrayambak> sent a code paste, see https://play.nim-lang.org/#pasty=gSDFknik
14:21:26FromDiscord<nocturn9x> oh
14:21:31FromDiscord<nocturn9x> it's another skill issue on my part it seems
14:21:40FromDiscord<nocturn9x> the usual off by one error
14:21:48FromDiscord<nocturn9x> I'm spawning 2 workers but only one is supposed to be spawned
14:21:56FromDiscord<nocturn9x> so worker #2 looks for data structures that do not exist
14:22:09FromDiscord<nocturn9x> (the first "worker" is actually the search main thread)
14:22:26FromDiscord<arkanoid> I have a constant string that I want to copyMem into a seq[byte] at runtime.↵If I do `copyMem(addr destBuffer[0], addr constString[0], len(constString))` I get `Error: expression has no address`. How to workaround this?
14:22:37FromDiscord<nocturn9x> `unsafeAddr`?
14:23:03FromDiscord<nocturn9x> would be helpful to know exactly which expression has no address
14:23:18FromDiscord<nocturn9x> likely the constant string, as it might be referenced in the code as an offset in the static section of the binary
14:24:07*ntat joined #nim
14:25:25FromDiscord<nocturn9x> k so the threads are getting spawned now but I'm getting really weird behavior
14:25:36FromDiscord<nocturn9x> it appears syncAll is blocking forever even though the workers stopped executing
14:25:43FromDiscord<nocturn9x> the main thread tells them to stop and they should abide by this
14:25:52FromDiscord<nocturn9x> it works with createThread so idk what's different here
14:25:57FromDiscord<xtrayambak> In reply to @arkanoid "I have a constant": I'd just copy the string, but that's wasteful
14:26:05FromDiscord<xtrayambak> In reply to @nocturn9x "`unsafeAddr`?": `unsafeAddr` is deprecated
14:26:09FromDiscord<nocturn9x> ah
14:31:57FromDiscord<nocturn9x> `Spawns the input function call asynchronously, potentially on another thread of execution.`
14:32:00FromDiscord<nocturn9x> tf you mean _potentially_
14:32:09FromDiscord<nocturn9x> nononononono
14:32:14FromDiscord<nocturn9x> it _has_ to go in a separate thread lmao
14:32:20FromDiscord<nocturn9x> I can't have it hog the main thread lol
14:32:36FromDiscord<nocturn9x> it'll fuck with everything
14:32:42FromDiscord<nocturn9x> that explains why stopping doesn't work correctly
14:32:51FromDiscord<nocturn9x> @mratsim?
14:33:52FromDiscord<nocturn9x> perhaps this was the showstopper last time
14:35:22FromDiscord<xtrayambak> In reply to @nocturn9x "tf you mean _potentially_": not-verbose-nuff documentation strikes again
14:35:28FromDiscord<xtrayambak> (edit) "In reply to @nocturn9x "tf you mean _potentially_": not-verbose-nuff documentation strikes again ... " added "and claims another victim"
14:35:48FromDiscord<nocturn9x> :kekw
14:35:54FromDiscord<nocturn9x> (edit) ":kekw" => "kek"
14:36:07FromDiscord<nocturn9x> why in the unholy fuck would I ever _not_ want to spawn the task in another thread
14:36:11FromDiscord<nocturn9x> how can it be asynchronous otherwise
14:36:19FromDiscord<nocturn9x> it says on the README it doesn't support async-await
14:36:27FromDiscord<nocturn9x> so how does it run the task if I don't call syncAll
14:36:27FromDiscord<nocturn9x> lmao
14:36:44FromDiscord<nocturn9x> number of times I wanted to murder the author of a thread library: +1
14:37:04FromDiscord<odexine> asynchronous means concurrent not parallel&
14:37:05FromDiscord<odexine> (edit) "parallel&" => "parallel"
14:37:11FromDiscord<nocturn9x> mate
14:37:12FromDiscord<xtrayambak> ^
14:37:18FromDiscord<xtrayambak> In reply to @odexine "asynchronous means *concurrent* not": ^
14:37:19FromDiscord<nocturn9x> sent a code paste, see https://play.nim-lang.org/#pasty=qvtQBVxj
14:37:24FromDiscord<nocturn9x> I am ACUTELY aware of the difference
14:37:27FromDiscord<nocturn9x> I write event loops for fun
14:37:35FromDiscord<nocturn9x> the guy writing this README probably is not, though
14:37:42FromDiscord<nocturn9x> since it literally says "parallel" right there
14:37:51FromDiscord<odexine> what library is this again
14:37:53FromDiscord<nocturn9x> so is it parallel or not
14:37:55FromDiscord<nocturn9x> nim-taskpools
14:38:03FromDiscord<nocturn9x> do I have to clone the library and fix the retardation myself smh
14:38:04FromDiscord<odexine> oh this one
14:38:06FromDiscord<xtrayambak> In reply to @odexine "what library is this": status IM's nim-taskpool
14:38:11FromDiscord<xtrayambak> I've never had a good time with it myself
14:38:15FromDiscord<nocturn9x> at least give me the option to force spawn it in a thread
14:38:18FromDiscord<xtrayambak> (edit) "nim-taskpool" => "nim-taskpools"
14:38:30FromDiscord<nocturn9x> having non changeable defaults should be grounds for death by firing squad
14:38:32FromDiscord<nocturn9x> genuinely
14:39:43*alexdaguy quit (Quit: w)
15:00:49FromDiscord<nocturn9x> yeah idk nim-taskpools is severely retarded
15:00:57FromDiscord<nocturn9x> I'm afraid it's trying to do something it's not supposed to
15:03:24FromDiscord<nocturn9x> like
15:03:28FromDiscord<nocturn9x> sent a code paste, see https://play.nim-lang.org/#pasty=gPMVENYN
15:03:32FromDiscord<nocturn9x> it doesn't even complete the spawning process
15:03:37FromDiscord<nocturn9x> it just dies
15:03:46FromDiscord<nocturn9x> and the traceback is mangled of course so it's useless garbage
15:04:13FromDiscord<nocturn9x> ehh altho
15:04:17FromDiscord<nocturn9x> sent a code paste, see https://play.nim-lang.org/#pasty=GqtBWYLc
15:04:21FromDiscord<nocturn9x> the if stmt is line 438
15:04:26FromDiscord<nocturn9x> so it is failing to load the atomic
15:04:31FromDiscord<nocturn9x> the question is why the fuck is it nil
15:05:03*jjido joined #nim
15:05:03FromDiscord<nocturn9x> since when can atomics even be nil
15:05:40FromDiscord<nocturn9x> can someone recommend a thread pool library that does not suffer from mental retardation and is still not trash to use
15:08:56FromDiscord<nocturn9x> even weave has this: spawn fnCall(args) which spawns a function that may run on another thread
15:09:07FromDiscord<nocturn9x> I don't want to _maybe_ run on another thread
15:09:12FromDiscord<nocturn9x> I want to run on another thread period
15:09:22FromDiscord<nocturn9x> genuine question: has anyone ever needed this?
15:09:32FromDiscord<nocturn9x> like why would I want to hog my main thread with random shit tasks
15:16:52FromDiscord<nocturn9x> @mratsim since weave is your library, please for the love of god give the option to always spawn a task on a worker thread
15:17:27FromDiscord<odexine> i think i understand now, its because if you only have one task in the queue why would you use a separate thread?
15:17:35FromDiscord<nocturn9x> because I want to????
15:17:39FromDiscord<nocturn9x> like give me the freaking option
15:17:57FromDiscord<nocturn9x> my engine was designed like this, I'm not about to rearchitecture the design because a library writer couldn't be arsed to expose a configuration option
15:18:30FromDiscord<nocturn9x> like is it so much to ask for something like Python's `ThreadPoolExecutor`
15:18:34FromDiscord<nocturn9x> it does one thing and it does it well
15:18:52FromDiscord<nocturn9x> who the fuck cares about a parallel raytracing demo if you can't actually direct the library to do the most basic shit?
15:19:20FromDiscord<nocturn9x> sorry for my tone but this is like the 340943578th time I deal with this shit and I'm tired
15:19:25FromDiscord<odexine> ok
15:19:29FromDiscord<nocturn9x> nim's threading ecosystem is a hot steaming pile of garbage
15:19:40FromDiscord<nocturn9x> just have one library that works in the stdlib smh
15:19:45FromDiscord<odexine> i mean simply i'll just stop interacting with you so dw about it
15:20:44FromDiscord<nocturn9x> `Weave can also be run as a background service and process jobs similar to the Executor concept in C++. Jobs will be processed in FIFO order.`
15:20:48FromDiscord<nocturn9x> is that maybe what I need
15:21:18FromDiscord<nocturn9x> well okay maybe but it looks way too overengineered
15:29:16*jjido quit (Quit: My laptop has gone to sleep. ZZZzzz…)
15:34:06FromDiscord<nocturn9x> welp it seems I ended up figuring out how to use `std/threadpools`
15:34:10FromDiscord<nocturn9x> will test now to see if anything breaks
15:34:29FromDiscord<nocturn9x> but holy guacamole is doing threading in nim a huge pain in the butt.
15:45:54FromDiscord<solitudesf> how do i unsubscribe from this blog?
15:49:57FromDiscord<nocturn9x> sent a code paste, see https://play.nim-lang.org/#pasty=jZyHyuZR
15:50:00FromDiscord<nocturn9x> #rip
15:50:05FromDiscord<nocturn9x> random crashes
15:50:44FromDiscord<nocturn9x> (the lines are all wrong)
17:20:06Amun-Rarandom? hmm, accessing uninitialized data?
17:29:00FromDiscord<nocturn9x> well
17:29:07FromDiscord<nocturn9x> I'm fairly sure it is initialized
17:29:11FromDiscord<nocturn9x> I could be wrong
17:29:16FromDiscord<nocturn9x> I'll check more thoroughly tomorrow
17:29:27FromDiscord<nocturn9x> I can only deal with so many multithreading headaches in one day
17:30:09Amun-Rahmm
17:31:29FromDiscord<nocturn9x> all I can say is it works fine for a while and then randomly crashes
17:31:36FromDiscord<nocturn9x> no issues when using threads spawned with createThreads
18:01:53*jjido joined #nim
18:17:45*bessieTheBoy joined #nim
18:19:56bessieTheBoyI know about a million people have probably asked this. But, I built nim on linux (Ubuntu/Debian) and I copied the binaries to my /usr/bin folder but when I try to do "nimble install sdl2" I get this error:
18:19:57bessieTheBoy Error: cannot open '/usr/lib/nim/lib/system.nim'
18:21:29Amun-Rahmm, weird, I install nim on debian manually except I don't pollute /bin directory and use /opt instead (I symlink all the /opt/nim/bin to /usr/local/bin tho)
18:22:02Amun-Raand I just clone the repo; git checkout v2.2.2; ./build_all.sh
18:22:42bessieTheBoysame :\ except polluting /bin dir (well technically /usr/bin)
18:22:49Amun-RaI promise myself I'll find a time to build a proper deb someday ;>
18:23:38Amun-Raah, I can see the error - you have to copy more dirs than just bin
18:23:57bessieTheBoyoh
18:24:24Amun-Raconfig and lib too
18:24:30Amun-RaI have those: bin compiler config dist doc icons lib tools
18:24:40bessieTheBoyk ill see if that fixes it
18:27:53Amun-RaError: internal error: genTypeInfo(tyNil)
18:28:24bessieTheBoynow I get:
18:28:24bessieTheBoyEvaluating as NimScript file failed with:
18:28:25bessieTheBoy        ... /tmp/nimblecache-1729675248220492780/sdl2_1755853463851935521/sdl2.nims(6, 8) Error: cannot open file: strutils
18:28:27bessieTheBoyoop
18:28:34Amun-RaI switched from 2.0.14 to 2.2.2 to see whether the same error occures as on 2.0.2; reverting to 2.0.14
18:28:49bessieTheBoyprobably should've done a pastebin for that, sorry
18:29:38Amun-Rarun "nim dump" and check all search paths
18:30:37bessieTheBoynim dump spits out the "list of currently defined symbols" is that correct?
18:31:13Amun-Rayes
18:31:17bessieTheBoyoh but at the bottom after "end of list" there's:
18:31:42Amun-Ranow it's a good idea to use pastebin ;)
18:31:46bessieTheBoy /usr/lib/nim/lib
18:31:54bessieTheBoyjust that
18:32:02bessieTheBoyoh
18:32:04bessieTheBoyyeah
18:32:32Amun-Rado you have std directory inside /usr/lib/nim/lib?
18:34:12bessieTheBoyyes
18:34:13bessieTheBoyalso
18:34:15bessieTheBoyhttps://pastebin.com/3BCUy5yv
18:35:06bessieTheBoya quick "ls | grep str" in /usr/lib/nim/lib/std shows that strutils.nim does not exist
18:35:46Amun-Rait's in "pure"
18:35:55Amun-RaI mean /usr/lib/nim/lib/pure
18:36:14bessieTheBoyjust realized that after a quick google search of "strutils.nim" XD
18:36:20Amun-RaI think you're missing /usr/lib/nim/config
18:36:24bessieTheBoyprob
18:36:29bessieTheBoylemme fix rq
18:36:42bessieTheBoybut how do I add that to search dirs?
18:37:04bessieTheBoyoh config will fix that i'd imagine?
18:37:05Amun-Rayou don't
18:37:07*xet7 joined #nim
18:37:09Amun-Rayes
18:37:42Amun-Raok, I reverted to 2.0.14; I'll play with 2.2.2's koch in spare time
18:38:24bessieTheBoyit works thx
18:38:31Amun-Rano worries
18:39:27Amun-Rahttps://github.com/nim-lang/Nim/blob/485b414fcec195bf217aba252e97c3c71f9bf489/config/nim.cfg#L47-L60
18:40:56bessieTheBoynice
18:41:07bessieTheBoythat makes sense
18:41:20Amun-Rathat was missing
18:41:26bessieTheBoyyeah
18:59:25*jjido quit (Quit: My laptop has gone to sleep. ZZZzzz…)
19:25:05*coldfeet joined #nim
19:26:19bessieTheBoybtw I took note on the deb thing you said and just made an amd64 deb.
19:26:32bessieTheBoyjust tested it in a chroot
19:26:38bessieTheBoychroot jail*
19:30:02bessieTheBoyonly downside is that I couldn't find if nimble works or not. I'm not good enough at chroot jails to add networking
19:31:01*jjido joined #nim
20:20:21*pbsds35 quit (Quit: The Lounge - https://thelounge.chat)
20:21:06*pbsds35 joined #nim
20:29:09*Jjp137 quit (Ping timeout: 248 seconds)
20:29:35*Jjp137 joined #nim
21:39:36*coldfeet quit (Quit: Lost terminal)
21:47:10*ntat quit (Quit: Leaving)
21:47:43*bessieTheBoy quit (Quit: Client closed)
21:53:08*rockcavera joined #nim
21:57:35*jjido quit (Quit: My laptop has gone to sleep. ZZZzzz…)
21:59:18FromDiscord<.bobbbob> If nim's std lib is mit licensed and statically compiled then does that mean pretty much any nim code will have to provide copyright info for nim?
22:05:03FromDiscord<Elegantbeef> No cause Nim libraries do not ship Nim code
22:05:50FromDiscord<Elegantbeef> In reality you can replace Nim's stdlib with your own stdlib if you so wish and license it under whatever you want
22:08:01*bessieTheBoy joined #nim
22:18:48FromDiscord<.bobbbob> I was under the impression that compiling mit licensed code from a library means you have to provide the copyright information for that library when you distribute it
22:19:09FromDiscord<.bobbbob> Maybe the info in the nimble file is enough?
22:20:13FromDiscord<.bobbbob> another question, how do you guys go about running tests in another test file for code that I dont want to export from that module? Say a function in the main file that's only supposed to be used in that file
22:50:54FromDiscord<.bobbbob> did nim 2.2.2 break nimble? it wont run any commands now, it just stalls
23:15:53*bessieTheBoy quit (Quit: Client closed)