<< 16-08-2019 >>
00:52:30*laaron joined #nim
01:19:04*endragor joined #nim
02:09:44*endragor quit (Remote host closed the connection)
02:50:40*lritter quit (Ping timeout: 246 seconds)
02:51:25*lritter joined #nim
03:04:59*laaron quit (Quit: ZNC 1.7.1 - https://znc.in)
03:05:29*laaron joined #nim
03:24:13*endragor joined #nim
03:32:45*endragor_ joined #nim
03:33:04*endragor quit (Read error: Connection reset by peer)
03:38:26FromGitter<kayabaNerve> ```code paste, see link``` ⏎ ⏎ :/ [https://gitter.im/nim-lang/Nim?at=5d562531029a51607fa118f2]
03:38:35FromGitter<kayabaNerve> Thankfully, it appears to be a fluke.
03:40:02FromGitter<kayabaNerve> Well. 1/4 runs so far...
03:44:25*laaron quit (Remote host closed the connection)
03:52:13*PrimHelios_ joined #nim
03:52:37*PrimHelios quit (Ping timeout: 258 seconds)
04:02:28*NimBot joined #nim
04:28:55*rayman22201 quit (Quit: Connection closed for inactivity)
04:33:53*rayman22201 joined #nim
04:41:11*rockcavera quit (Remote host closed the connection)
04:49:33*nsf joined #nim
05:35:00*terps joined #nim
05:42:55*narimiran joined #nim
06:21:24*PMunch joined #nim
06:32:28*terps quit (Ping timeout: 264 seconds)
06:44:25*solitudesf joined #nim
06:48:29*shomodj joined #nim
06:48:55*rayman22201 quit (Quit: Connection closed for inactivity)
07:00:00*gmpreussner quit (Quit: kthxbye)
07:03:39*krux02 joined #nim
07:04:42*gmpreussner joined #nim
07:46:54FromGitter<alehander42> ohhow does it happen ?
07:48:48*shomodj quit (Ping timeout: 245 seconds)
07:55:52*shomodj joined #nim
08:00:04*shomodj quit (Ping timeout: 246 seconds)
08:06:25*shomodj joined #nim
08:07:01FromGitter<kayabaNerve> It's some async bug. It happened almost immediately on boot but I'm working with automated testing.
08:07:46FromGitter<kayabaNerve> I added a print statement to the try catch because an Exception was being triggered, and catching the exception of an async function from within an async function...
08:08:13FromGitter<kayabaNerve> Anyways. Codebase is unfortunately too large for me to try to create a MWE. Didn't even bother creating a commit gist combo... probably should've.
08:10:45*shomodj quit (Ping timeout: 244 seconds)
08:20:58*terps joined #nim
08:24:19*floppydh joined #nim
08:44:12*actuallybatman quit (Ping timeout: 245 seconds)
08:45:32*shomodj joined #nim
08:52:16*sammich quit (Quit: https://quassel-irc.org - Chat comfortably. Anywhere.)
08:53:25*abm joined #nim
08:53:27*sammich joined #nim
09:02:25*chemist69 joined #nim
09:13:14*ng0 joined #nim
09:14:12*terps quit (Ping timeout: 245 seconds)
09:19:48*absolutejam joined #nim
09:21:22*shomodj quit (Quit: My MacBook has gone to sleep. ZZZzzz…)
09:28:47*terps joined #nim
09:50:19*shomodj joined #nim
09:50:55*shomodj_ joined #nim
09:54:53*shomodj quit (Ping timeout: 258 seconds)
10:15:03*terps quit (Ping timeout: 245 seconds)
10:28:17*hugogranstrom joined #nim
10:37:18livcdhttps://szibele.com/memory-footprint-of-gui-toolkits/chart.svg
10:41:59*stefanos82 joined #nim
10:48:00kungtotteThe biggest issue with Electron apps isn't their memory use, it's that every app ships their own Electron.
10:52:12*hugogranstrom quit (Ping timeout: 272 seconds)
10:58:07*terps joined #nim
11:14:49Araqthe biggest issue with Electron is that immediately you inherit all the browser's security problems
11:18:48*terps quit (Ping timeout: 272 seconds)
11:33:12FromGitter<arnetheduck> given that a fair amount of effort has gone into the security of electron, it might very well be that running inside of electron might on average be better than whatever code you're able to produce yourself, and the sandbox will protect you from your own *mostly bad* code
11:37:16Araqdepends. if your app doesn't need network connections the browser is the last thing to base your UI on
11:43:54*clyybber joined #nim
11:48:15*terps joined #nim
11:54:42*rockcavera joined #nim
12:09:27*terps quit (Ping timeout: 250 seconds)
12:11:06FromGitter<alehander42> after 5 years electron will run wasm on gpu in ring 0 in electron-os and we will be all finished
12:11:24FromGitter<alehander42> (this, but half-seriously)
12:12:33FromGitter<mratsim> There is WebGPU
12:12:58FromGitter<mratsim> https://en.wikipedia.org/wiki/WebGPU
12:21:03*Kaivo joined #nim
12:36:49Araqalehander42: btw https://github.com/pragmagic/karax/pull/110
12:37:01Araqshould fix the known performance problems
12:40:17FromGitter<alehander42> awesoe!
12:40:26FromGitter<alehander42> awesome*
12:40:41FromGitter<alehander42> i am using a fork based on an old version of the diff algorithm
12:40:57FromGitter<alehander42> but i'll try this one these days
12:41:20*nsf quit (Quit: WeeChat 2.4)
12:41:37*shomodj joined #nim
12:42:05FromGitter<alehander42> does toDom work now
12:42:25FromGitter<alehander42> i am using vnodeToDom in several places (where i need to dynamically pass html created by a view)
12:42:52*absolutejam quit (Ping timeout: 246 seconds)
12:43:21FromGitter<alehander42> what i mean is, does it work in the same way: do i need to useAttachedNode when i manually use it?
12:43:47FromGitter<alehander42> it seems it just caches it ok
12:45:01*endragor_ quit (Remote host closed the connection)
12:45:34*shomodj_ quit (Ping timeout: 272 seconds)
12:47:13*absolutejam joined #nim
12:53:10Araqalehander42: vnodeToDom still exists and behaves as before
12:53:58FromGitter<alehander42> ok
12:54:12FromGitter<alehander42> btw i run usually several instances of karax
12:54:47FromGitter<alehander42> as i use a resizable layout engine and put a karax instance in each panel
12:55:07FromGitter<alehander42> maybe this helps with perf too, as they work on smaller trees
12:55:21FromGitter<alehander42> only the DOM of their panel
13:00:47*shomodj quit (Quit: My MacBook has gone to sleep. ZZZzzz…)
13:00:49*seni_ joined #nim
13:00:52Araqwell it's backwards compatible except that 'n.dom' access can fail now
13:00:58Araqif you refer to an old 'n'
13:01:07Araqmaybe I'll undo this change though.
13:01:13Araqit was inspired by --newruntime :P
13:01:21Araqno crazy .dom sharing
13:02:13*shomodj joined #nim
13:08:02*terps joined #nim
13:42:32FromGitter<bevo009> Can I get a strformat ninja to check this out please? ⏎ This strformat statement compiles at top level: https://play.nim-lang.org/#ix=1RSU ⏎ But the same line in a template fails: https://play.nim-lang.org/#ix=1RT1 ⏎ To work, I have to replace:' ⏎ ``` let elapsedStr = &"{elapsed:.3f}"``` ... [https://gitter.im/nim-lang/Nim?at=5d56b2c71dadc42a117fb1b2]
13:42:54clyybberAraq: WDYT about merging all nk...Expr with nk...Stmt, aside from the bootstrapping issues?
13:53:59FromGitter<bevo009> Nevermind, fixed it with {.inject.} ...https://play.nim-lang.org/#ix=1RT7
13:54:16FromGitter<bevo009> Hope that's the right method to use
13:55:18Araqclyybber, this morning I implemented the string based nnk* comparisons in the VM
13:55:26Araqbut then I stopped and considered the full situation
13:55:49*dddddd joined #nim
13:55:55Araqand rather than "fixing" the node kinds in slightly incompatible ways
13:56:18AraqI think we must stop these subtle AST changes
13:56:57clyybberAraq: Why?
13:57:27AraqIMO we need an ir.nim module that simply replaces macros.nim
13:57:51clyybberFair enough, but that shouldn't stop us from fixing AST inconsistencies, right?
13:58:14Araqsee also https://forum.nim-lang.org/t/5097 for why NimNodes are kinda bad :-)
13:58:40Araqno, that's exactly the point. don't fix "inconsistencies"
13:58:53Araqdon't break somebody's macro
14:00:54clyybberMacros that rely on the disctinction between something like ElifExpr or ElifBranch are already broken anyways.
14:01:45clyybberSo maybe with aliasing nnkElifExpr and nnkElifBranch to nnkElif and mergine nkElifExpr and nkElifBranch we could retain sane backwards compatability
14:02:10*terps quit (Ping timeout: 252 seconds)
14:02:41Araq"already broken anyways" well if they worked before and don't afterwards
14:02:49Araqthen *we* broke them
14:03:20Araqand nnkElifBranch vs nnkElifExpr is just a tiny detail, the AST is *full* of downright undocumented node kinds
14:05:10clyybberWe have tests for important packages right? And if we document in the changelog that instead of relying on the nnk...Stmt vs Expr disctinction one should check the type, I think it would be fine.
14:06:24*shomodj quit (Quit: My MacBook has gone to sleep. ZZZzzz…)
14:06:28clyybberI'm just talking about Stmt vs Expr node kinds ATM, since that distinction is IMO useless cruft, if we can just check the type instead
14:06:41Araqthe important packages are incomplete and since the ast is quirky anyway, why bother
14:08:10*shomodj joined #nim
14:08:20clyybberTo make life more easy?
14:08:49clyybberAs in simpler macros, less weird edge cases, and (neglible) a more simple compiler
14:10:25*shomodj quit (Read error: Connection reset by peer)
14:10:48*shomodj joined #nim
14:11:48*terps joined #nim
14:14:23*PMunch quit (Remote host closed the connection)
14:16:16FromDiscord_<Kiloneie> Could i get pros and cons of Nim thhings you like and hate the most ? I need more grounded information for the video i am working on.
14:23:13*lritter quit (Quit: Leaving)
14:41:08Araqclyybber, I'm not buying it, it's such a minor thing, we should focus on the important stuff
14:43:16*terps quit (Ping timeout: 264 seconds)
14:44:44*floppydh quit (Quit: WeeChat 2.5)
14:47:45clyybberAraq: IMO it's important too, though I agree not as important as some other stuff
14:53:42FromDiscord_<Kiloneie> Nim runs a garbage collector by default right ? How fast is it compared to like C# one ?
14:54:37Araqmuch slower but it's support "realtime" and Nim is easier to optimize to avoid allocations
14:54:55Araqalso with --newruntime the GC is gone
14:57:11FromDiscord_<Kiloneie> Do all the algoritms support realtime ? on and off at a whim ?
14:57:35*terps joined #nim
14:58:03Araqwhat does that mean?
14:59:04FromDiscord_<Kiloneie> Okay my bad, different algoritms, one of them is a real time one.
14:59:20Araqyup
15:00:17clyybberAraq: Should I move implicitlyDiscardable from semstmts to ast.nim ? I need it in injectdestructors
15:02:02FromDiscord_<Kiloneie> Nim's main page does a pretty good job at introducing nim, but it does miss a few important features that are then listed in Feratures tab, and still not all of them ? A lot of digging xD.
15:02:04livcdNim's GC is much slower compared to C# ?
15:02:37Araqlivcd, depends on the workloads
15:03:04Araqallocation heavy code is usually served by a bump-pointer allocator which we don't really have
15:04:14Araqbut it's pretty much universal that in most modern languages you need to optimize the number of allocations and C# doesn't make that easy
15:04:44Araqit's getting better in C# but it's not where Nim is
15:05:02FromDiscord_<Kiloneie> C# also only has that one GC, Nim has more, or none at all.
15:05:22AraqNim can use Go's GC btw. Never tried to use that option
15:05:50FromDiscord_<Kiloneie> Wait, how does one turn it entirely off ? That is in the new runtime only ?
15:07:04Araq--gc:none works since forever
15:07:17Araqwith --gc:none you either:
15:07:25Araq- use a restricted subset of Nim
15:07:39Araq- don't mind that only on process exit the OS frees the memory for you
15:08:10FromDiscord_<Kiloneie> I know that sequences are GC., How do you handle that once you turn it off ?
15:08:29Zevvyou dont. it just eats up and never frees
15:08:34Araqas I said, either you don't use them or you live with them leaking memory
15:09:07FromDiscord_<Kiloneie> Does the newruntime solve that ?
15:09:13Araqyup
15:09:32FromDiscord_<Kiloneie> Will Nim be even faster then ?
15:09:46FromGitter<erhlee-bird> what's the general usecase for --gc:stack?
15:10:16Araqkiloneie: depends on the program.
15:10:36Araqas far as we can tell from this point, usually it is faster
15:11:42Araqerhlee-bird: --gc:stack is pretty dead, --newruntime does it better
15:11:43livcdI hope for much easier parallelism story
15:12:21FromDiscord_<Kiloneie> Okay. If i were to say Nim's standard library can be used for vast majority of programs one will write, will i be correct or very wrong ? I know it's easy to bind/wrap C libraries and that solves everything, but without that how extensive is it ?(i did read quite some on it, but i can't really tell, besides no GUI library and drawing ?)
15:12:51leorizeif you don't do gui stuff, then yea, it should be fine
15:13:20CcxWrkI don't consider the async parts to be particularly ready.
15:13:37FromGitter<bevo009> https://robert-mcdermott.gitlab.io/posts/speeding-up-python-with-nim/
15:13:59CcxWrkOther than that, it seems to cover most of the core OS API and bunch of useful data structures.
15:15:38CcxWrkbevo009: Does not even mention Cython. But nice.
15:16:20AraqKiloneie: the stdlib has good parts but many complain about it. You can also find Nimble packages that work better than the stdlib's stuff, but that's not unique to Nim.
15:16:35*seni_ quit (Quit: Leaving)
15:17:05*terps quit (Ping timeout: 250 seconds)
15:19:03FromDiscord_<Kiloneie> It does scream bigger community the fact that Nimble stuff works better.
15:20:41Araqsomebody should take important_packages.nim and turn it into a Nim distribution
15:21:09Araqso ... any Windows experts around?
15:21:32FromDiscord_<Kiloneie> Yeh, thats what i meant with bigger community xD... stuff like that should of been done already. Soon enough.
15:24:00livcdWhat windows expertise is required ?
15:25:11Araqunicode and terminals
15:25:47*terps joined #nim
15:28:04AraqI'm trying to fix https://github.com/nim-lang/Nim/issues/11618 for good but it seems impossible
15:28:22Araqnot even WriteConsoleW does the trick
15:30:40*terps quit (Ping timeout: 264 seconds)
15:31:43FromDiscord_<Kiloneie> should i say these 3 words on video : Efficient, expressive, elegant ? Or is saying that it performs on par with C whilst looking like Python better... or both ?
15:34:14Araqjust say it your way
15:34:52Araq"it combines C-like speed (and by that I mean C-like, not a factor 2 slower) with Python's ease on the eyes"
15:35:34FromGitter<Varriount> Araq: What about https://docs.microsoft.com/en-us/windows/win32/api/stringapiset/nf-stringapiset-multibytetowidechar with different code pages?
15:38:36FromGitter<Varriount> Or rather, https://docs.microsoft.com/en-us/windows/win32/api/stringapiset/nf-stringapiset-widechartomultibyte
15:39:01FromGitter<Varriount> Couldn't you use WideCharToMultibyte with the current code console code page?
15:39:29AraqVarriount: ok, can try that instead. good idea
15:39:33Araqthing is
15:40:02Araqthe Console now defaults to 65001 (UTF-8)
15:40:11Araqon my Win 10 machine and still nothing works
15:40:18FromDiscord_<Kiloneie> I wrote down on par with C(performance, from what iread online it's C speed there, and a bit less in some other areas, basically C... idk.)
15:40:53FromDiscord_<Kiloneie> It compiles so fast with such helpful tracebacks, it almost faster to write code in it than in Python lol.
15:41:48FromDiscord_<Kiloneie> I remember using Dev-C++ at school years ago, that was real helpful when it crashed, nothing, nada. (oh wait a semicolon is missing -.-)
15:44:39clyybberAraq: What was injectdefaultcalls purpose?
15:46:00clyybberAnd will we ever need it in the future?
15:46:08Araqclyybber, optimization, only produce 'var x = default(T)' implicitly when the control flow analysis says we need it
15:46:16AraqI hope we can use it in the future
15:46:22clyybberAh cool, sounds nice
15:47:09livcdi have no idea how it works but i had an issue displaying korean/chinese chars in windows cmd without doing smh explicit
15:47:40Araqlivcd, for chinese you also need to select the right font
15:47:47Araqbut I never got it to work either.
15:47:55Araqhowever, I'm testing German umlauts for now
15:48:26livcdoh
15:50:40Araqhmm echo getCurrentEncoding() says windows-1252
15:50:40shashlickhas anyone used the Intel compiler with Nim
15:51:19clyybberI suppose mratsim has at some point
15:51:39FromGitter<zacharycarter> yeah - I'm pretty sure someone has
15:52:12FromGitter<zacharycarter> I just don't know who :P
15:52:22FromGitter<Varriount> I know Demos did
15:52:42FromGitter<zacharycarter> I don't know if I want to spend time debugging HCR more, or just work on my game
15:52:59FromGitter<zacharycarter> lua so far is working for me - but working HCR would be very nice
15:53:05Araq"more"? what did you try?
15:53:30FromGitter<zacharycarter> well - I wasn't trying fixes yet - I was just trying to figure out what exactly was causing the first set of errors I was encountering
15:54:01FromGitter<zacharycarter> basically `failedAssertImpl` is not being found in the list of registered procs
15:54:55FromGitter<zacharycarter> it was a bit difficult to figure out why exactly that was happening though - because procs get registered for the module you're compiling as well as for Nim stdlib modules as well
15:55:21FromGitter<zacharycarter> and it seemed the call to register and get this proc specifically were lining up correctly
15:55:49FromGitter<zacharycarter> this definitely won't fix the reading from beyond the top of the stack issue - but I figured if I can figure out what's causing this one, then I can work on that one
15:59:56FromDiscord_<Shield> oh wow, seqs and strings are really problematic, if you chose to ditch them, you basically can't use most if not all libs
16:01:11FromGitter<zacharycarter> why would you choose to ditch them?
16:01:54disruptekpesky strings.
16:02:21FromDiscord_<Shield> they seem to be the source of all evil, it makes it really hard to use other gcs
16:02:25disruptekmeaningless seqs.
16:03:44FromGitter<zacharycarter> which gc's? I've never heard of anyone complaining about using boehm with strings / seqs
16:03:56FromGitter<zacharycarter> gc none of course - but it's pretty obvious why that's the case
16:04:09FromGitter<zacharycarter> regions maybe?
16:04:13FromDiscord_<Shield> I still think that a function which takes a memregion for object creation will solve a lot of stuff
16:04:19FromDiscord_<Shield> yeah regions too
16:04:49Araqregions are viral. so everything that needs memory grows this "region" parameter
16:05:23Araqand it bubbles up for everything and in the end you use *more* memory than with alloc/dealloc
16:05:45AraqI researched these things quite a bit, you know
16:06:06Araqyou're better off with destructors/deallocs
16:07:06Araq--newruntime came after --gc:regions so maybe somebody learned something from it
16:07:30*shomodj_ joined #nim
16:07:45FromDiscord_<Shield> is it not possible to copy variables from one region to the other without some hacks? the allocator using one and only region makes any object that grows do funny things
16:09:32FromDiscord_<Shield> doesn't newruntime still use gc for some stuff? or is it fully turned off?
16:09:56FromGitter<zacharycarter> so when compiling nimhcr with `-d:traceHcr` `get proc: C:\Users\carte\nimcache\test_d/stdlib_assertions.nim.c.dll failedAssertImpl_W9cjVocn1tjhW7p7xohJj6A` is the last line before the crash but there is no corresponding `register proc` for that proc - that's what I'm debugging atm
16:10:15FromGitter<zacharycarter> basically something is up with the codgen here
16:10:31clyybberShield: For unowned refs it still uses refcounting.
16:10:51shashlickAraq: is it possible to use the same region across main exe and dlls?
16:10:53*shomodj quit (Ping timeout: 245 seconds)
16:11:13shashlickthat might solve the multiple-gc issue right?
16:11:20FromDiscord_<Shield> shashlick yeah you can pass it normally
16:11:27FromGitter<zacharycarter> but newruntime will do this
16:11:33clyybberShield: Though there isn't a gc running. It's just that the destructor of an unowned ref checks the refcount before doing anything.
16:12:14FromDiscord_<Shield> but you're really giving your region to the dll, that's a lot of trust when you can only deallocate the whole region or you don't
16:12:33FromDiscord_<Kiloneie> How does one explain operator overloading to a mere mortal ?
16:12:42FromGitter<zacharycarter> google
16:12:52shashlickdo we have any docs or examples of regions
16:12:56FromGitter<Varriount> Think of operators like procedures with fancy names?
16:13:08FromGitter<zacharycarter> there's nothing special about operator overloading - it's the same as overloading anything else
16:13:24FromDiscord_<Shield> only the main gc is documented at all
16:13:25FromGitter<zacharycarter> key is remembering the difference b/w overloading and overriding
16:13:29disruptekfirst, find yourself a mortal...
16:13:52FromGitter<zacharycarter> I don't see the point of focusing on gc:regions when newRuntime will solve all of this
16:14:22shashlickregions or newruntime, without examples, i'm going to be stuck regardless
16:14:23disruptekthey solve different problems.
16:14:48FromGitter<zacharycarter> I mean that they will allow what shashlick is struggling with
16:15:03FromGitter<zacharycarter> there is a lot of example code / explanations around newruntime
16:15:06FromGitter<zacharycarter> but it's still in development
16:15:23FromGitter<zacharycarter> so they may be outdated
16:15:37FromGitter<zacharycarter> newruntime is better documented than gc:regions
16:15:49shashlickif i can use regions to simply tell all the gcs to work in one area, that's just like one gc
16:16:14FromDiscord_<Shield> region is very straight forward, just read the source code, there's a main region, and you can create more regions, you use it by having a block inside a macro to switch regions, the problem is, all allocations happen in the current region alone, god forbids you trying to pass a variable from one region to the other
16:16:41FromDiscord_<Shield> with seq growing and reallocating, it'll jump around if you don't pay attention
16:17:09FromDiscord_<Shield> there's also some hack in it to deal with seq internally but I never understood it
16:17:13Araqseqs growing works with regions, the seq remembers its region
16:17:44shashlickwell, if it is as easy as creating a region in the main exe and passing it to the dlls, it seems like a simple fix for the near term
16:18:27FromDiscord_<Shield> it fails when the seq is over a certain size and I don't know why, if you want to do a copy, you have to do a region within a region
16:19:10FromDiscord_<Shield> wouldn't it be simpler to just pass the region as a parameter so the allocator never have to guess? it will give more control
16:19:20*hugogranstrom joined #nim
16:20:05AraqShield: https://github.com/nim-lang/Nim/pull/11920
16:20:15Araqso people are working on --gc:regions
16:20:27FromDiscord_<Shield> tbqh after reading about newruntime multiple times I cannot understand it without proper examples
16:20:53disruptekyeah, it was leorize and Shield that instigated that patch.
16:21:04disruptekthis topological superconductivity paper is insane.
16:21:05Araqoh I didn't know that :p
16:21:12shashlickokay so what should i look for in the docs to figure out regions
16:21:53FromDiscord_<Shield> there's nothing about it in the docs
16:22:36*hugogranstrom quit (Client Quit)
16:22:42FromDiscord_<Shield> I'll try to write an example with most features
16:23:05FromGitter<zacharycarter> where is `BModule` defined?
16:23:20shashlickthanks Shield
16:23:33FromGitter<zacharycarter> I can only see it in the JS backend
16:23:52FromGitter<zacharycarter> nm found it
16:25:37*abm quit (Quit: Leaving)
16:26:43shashlickhow come devel docs doesn't show withRegion
16:29:29Araqshashlick, docs always assume --gc:refc (the default)
16:30:13shashlickthis is similar to how osseps was broken then
16:32:13*Ven`` joined #nim
16:32:58FromGitter<Varriount> Araq: If you're still having problems with console encoding, you could see how mingw's printf is implemented
16:35:25shashlickOne other question - is it possible to get a list of exported procs from a dll at runtime
16:35:30shashlickAfter dynlib
16:35:38shashlickOr some other way
16:36:01AraqVarriount: Now it's completely weird, only *some* Umlauts are rendered correctly
16:36:54*terps joined #nim
16:37:33Araqshashlick, I use "CEF explorer" for that
16:37:46FromGitter<zacharycarter> well - the reason that `failedAssertImpl` procs are not being registered with HCR is because the `isReloadable` check fails
16:37:53Araq"CFF explorer"
16:38:02FromGitter<zacharycarter> and that check is - `m.hcrOn and sfNonReloadable notin prc.flags`
16:38:27Araqwhy does it need to be able to "reload" this crap?
16:38:37Araqit's just assert, it doesn't change later
16:38:46FromGitter<zacharycarter> that's a good question....
16:38:46*shomodj_ quit (Quit: My MacBook has gone to sleep. ZZZzzz…)
16:39:18FromGitter<zacharycarter> I guess because the code I'm reloading calls assert - so maybe it's because of that?
16:39:29FromGitter<zacharycarter> still doesn't make much sense though
16:39:42AraqVarriount: My string must start with an ascii char and then an umlaut may follow
16:39:51Araqthen the umlaut is rendered
16:39:59Araqbut not if I start the string with an umlaut
16:40:36AraqI'm beginning to think this a new Windows bug
16:40:59FromGitter<Varriount> What happens if you change your code page to a non-utf8 CP?
16:41:12FromGitter<Varriount> Or the example in the issue?
16:41:28*terps quit (Ping timeout: 264 seconds)
16:42:45FromGitter<Varriount> Any idea what the V lang implementation does?
16:42:57Araqit uses printf
16:43:11Araqbut now everything works here
16:43:20Araqif I give the string an ASCII prefix
16:43:50Araqso it just works (TM) on new Windows 10
16:43:59Araqfor strings that start with "A"
16:44:17Araqgood enough for me, I always wanted to start all my output with "A"
16:44:50Araqtry it yourself
16:44:52Araqecho "A你好"
16:44:53Araqvs
16:44:56Araqecho "你好"
16:45:00FromGitter<Varriount> What about printf? Does printf work without the ascii character?
16:46:36Zevvmaybe you should start all strings with A followed by a backspace "A\b..."
16:47:22clyybberAraq: Now make all strings start with Araq
16:47:38clyybberNim, developer signed edition
16:50:10*terps joined #nim
16:52:43disruptek🤣
16:55:13FromDiscord_<Shield> @shashlick https://play.nim-lang.org/#ix=1RU6
16:55:37FromDiscord_<Shield> make sure to use this pr to fix a bug, otherwise it doesn't free memory properly https://github.com/nim-lang/Nim/pull/11920
16:55:55FromGitter<zacharycarter> so for `assertions.nim` `hcrOn` is false - I'm not sure what sets this flag per module
16:57:04*terps quit (Ping timeout: 264 seconds)
16:57:47FromDiscord_<Shield> it's kinda tricky with seq and string, you should note that all allocations can happen in one region only, and you have to deallocate the whole region in case there's a memory leak
16:58:21FromDiscord_<Shield> I don't think you can trust dlls with it
16:59:13FromDiscord_<Shield> it also consumes twice the memory when you copy stuff around between regions if you're not careful
17:00:55FromGitter<zacharycarter> is @onqtam active in Nim at all anymore, or @zah ?
17:02:25FromDiscord_<Shield> is there any clear examples for newruntime?
17:02:35FromGitter<zacharycarter> the tests I guess
17:03:12FromGitter<zacharycarter> ah - I think I know possibly why the issue is happening
17:04:30FromGitter<zacharycarter> there's a proc called `hcrOn*(conf: ConfigRef)` which checks to see if the module config global options contains `optHotCodeReloading`
17:04:59FromGitter<zacharycarter> so if assertions.nim is not compiled with - `-d:hotCodeReloading` - this check will fail
17:05:23FromGitter<zacharycarter> but the larger question you asked Araq remains - why does this need to be a registered proc to begin with?
17:06:07FromDiscord_<Shield> are those tests available in 0.20.2? which folder?
17:06:20clyybberShield: tests/desctrucor
17:06:26clyybberdestructor
17:06:37FromGitter<zacharycarter> it's weird though - a lot of other `stdlib` modules have procs registered - so why this specific one fails...
17:07:42*Ven`` quit (Quit: My MacBook has gone to sleep. ZZZzzz…)
17:08:20*Ven`` joined #nim
17:08:40FromGitter<alehander42> @zacharycarter zah is very active, onqtam is a bit busy probably tho
17:08:49FromGitter<alehander42> is it something about hcr?
17:08:56FromGitter<zacharycarter> mmhmm
17:10:13FromGitter<alehander42> ah, i see
17:10:25*terps joined #nim
17:11:05FromGitter<alehander42> but why is it bad that
17:11:12FromGitter<alehander42> those assertions are hcrOn
17:11:43FromGitter<zacharycarter> it's not that they're bad - it's that assertions.nim fails this check while other stdlib modules pass it
17:12:10FromGitter<zacharycarter> so the function pointer to assertImpl is never registered with the HCR table of procedures
17:12:36FromGitter<zacharycarter> and the HCR module tries to get this function pointer and that causes an error in tables.nim
17:12:46FromGitter<zacharycarter> I assume because I am using assertions in my code that is being hot reloaded
17:13:07*ng0 quit (Quit: Alexa, when is the end of world?)
17:13:57*Ven`` quit (Quit: My MacBook has gone to sleep. ZZZzzz…)
17:15:34*ng0 joined #nim
17:17:22*shomodj joined #nim
17:18:23AraqShield: you compile your code with --newruntime -d:useMalloc
17:18:42Araqand it it compiles, you're good. if not you need to patch the code/stdlib...
17:18:51*terps quit (Ping timeout: 250 seconds)
17:20:27AraqVarriount: it gets ever more weirder, 'printf' works indeed
17:22:06Araqso it's 'fwrite'
17:28:04FromGitter<zacharycarter> not trying to be too controversial here - but I only see one mention of `proof of concept` in the issue about Nim HCR
17:28:13FromGitter<zacharycarter> the one that announced the grant
17:28:20disruptekalways nice to pull and see nim bugs fixed; thank you. 😍
17:28:31FromGitter<zacharycarter> but I guess it doesn't matter - what was delivered is what was delivered
17:29:32Araqso, printf works, writeConsole doesn't
17:29:50Araqwhat does writeConsole wrong, usually using the WinAPI directly fixes everything
17:30:36*actuallybatman joined #nim
17:32:07FromGitter<Varriount> Araq: I think (I'm not certain) that printf is calling mbtow as part of the conversion process
17:34:24FromGitter<Varriount> Lets see... printf calls pformat, which calls fputc
17:34:34FromGitter<alehander42> @zacharycarter
17:34:48FromGitter<zacharycarter> yo
17:34:55FromGitter<alehander42> so the thing is that tables.nim doesnt work with hcr/
17:35:04FromGitter<alehander42> ?* trying to get a simple way to repro
17:35:26FromGitter<zacharycarter> I don't think it's an issue with tables.nim though
17:35:33FromGitter<zacharycarter> but that's also unfortunate it does not
17:35:53FromGitter<zacharycarter> HCR upon initialization loads a bunch of function pointers into a table
17:36:19FromGitter<zacharycarter> the error I'm facing is caused by the fact that the function pointer for `failedAssertImpl` isn't ever added to that table
17:36:37FromGitter<zacharycarter> I think maybe the reason it doesn't work with tables is because of another issue, which we might have an inkling of a clue about
17:36:51FromGitter<zacharycarter> caused by reading memory beyond the top of the stack
17:38:04Araqah!
17:38:18AraqI cannot use writeConsole anyway since that doesn't do IO redirections
17:39:00Araqshould use WriteFile
17:39:08Araqwhich is broken too, Jesus...
17:39:10Araqbbl
17:42:11*shashlick quit (Remote host closed the connection)
17:42:57*shashlick joined #nim
17:43:05*rayman22201 joined #nim
17:44:12FromGitter<alehander42> @zacharycarter but cant you compile assertions with hcrOn somehow
17:44:30FromGitter<zacharycarter> compilation works fine
17:44:35FromGitter<zacharycarter> runtime crash
17:44:49FromGitter<zacharycarter> oh I see what you mean
17:44:56FromGitter<zacharycarter> compile that single module with that flag - let me try that
17:45:38FromGitter<alehander42> yes
17:46:01FromGitter<alehander42> or find out why it isnt compiled with it on
17:46:20*Trustable joined #nim
17:46:46FromGitter<zacharycarter> well - that's why I'm trying to figure out but it's not straightforward
17:46:51FromGitter<zacharycarter> at least not to me
17:46:54shashlick@Shield - so if a var goes out of scope, its memory is not released till the region is freed?
17:47:44rayman22201Not shield, but yes @shashlick
17:48:08rayman22201Good morning all
17:48:15FromGitter<zacharycarter> o/
17:48:23FromGitter<zacharycarter> ugh - it's almost 9PM here haha
17:48:33clyybberSame, good morning rayman
17:48:48FromGitter<zacharycarter> clyybber: where are you at?
17:48:55rayman22201Lol. Aren't timezones fun.
17:49:44FromGitter<zacharycarter> it's still light out here though!
17:50:18FromGitter<zacharycarter> it will probably get dark around 10
17:51:12rayman22201It's all a lie. #flatearth :-P j/k
17:51:46clyybberzacharycarter: Germany, I just noticed its 8PM not 9 lol
17:52:05FromGitter<zacharycarter> :P I was about to say! You must be in my time zone
17:52:07shashlickwell then it isn't as useful to use a region
17:52:31shashlickyou cant have long running regions then
17:52:34FromGitter<zacharycarter> shashlick: we should just combine our forces and figure out HCR
17:52:39FromGitter<zacharycarter> and why it's borked
17:53:00FromGitter<zacharycarter> not sure how easy it will be though
17:56:17disruptekyou know you won't have to twist shashlick's arm. 😜
17:56:25clyybberI'll join your forces yoo
17:56:31clyybbers/yoo/too
17:56:35FromGitter<zacharycarter> force team assemble
17:56:35disruptekoh shoot!
17:56:39clyybberI have dabbled quite a bit in HCR
17:56:48clyybberseems like I didn't use asserts tho
17:56:57FromGitter<zacharycarter> I think there are a number of HCR issues
17:57:13Zevvis the feature worth the complexity?
17:57:25clyybberIMO yes
17:57:32FromGitter<zacharycarter> this is the issue I filed which I think would be a good starting point though: https://github.com/nim-lang/Nim/issues/11718
17:57:33clyybberBecause it actually isn't that complex
17:57:51FromGitter<zacharycarter> it would also be a separating feature for Nim
17:58:13FromGitter<zacharycarter> someone familiar with the compiler (aka you clyybber) would also make debugging this 100x easier I imagine
17:58:22Zevvtrue
17:59:07FromGitter<Varriount> I don't know - to me, HCR seems only good for certain niche features.
17:59:10FromGitter<zacharycarter> from what I've dug into so far - once asserts are no longer an issue - then there's an issue about memory being read from beyond the top of the stack
17:59:25FromGitter<zacharycarter> which Araq thought might be due to the gc having bad pointers
18:00:02FromGitter<zacharycarter> @Varriount - how many times though have people inquired about plugin systems / scripting / etc.. in Nim?
18:00:08FromGitter<zacharycarter> my guess is in the hundreds of times
18:00:59Zevvbut that is not what hcr is for
18:01:15Zevvits main goal is shorten development roundtrip times
18:01:24*krux02 quit (Remote host closed the connection)
18:01:24FromGitter<zacharycarter> which is the point of the above as well
18:01:33FromGitter<zacharycarter> you write scripts so you can reload them at runtime
18:01:38FromGitter<zacharycarter> you author plugins so you can reload them at runtime
18:01:52*shomodj quit (Quit: My MacBook has gone to sleep. ZZZzzz…)
18:02:00Zevvyes, but that is all not hcr. you need the compiler at hand for hcr
18:02:15disruptekthis.
18:02:34FromGitter<zacharycarter> you need nimhcr.dll
18:02:38FromGitter<zacharycarter> and nimrtl.dll
18:02:49disruptekwhere does the blob come from, though?
18:04:06FromGitter<zacharycarter> there is no live compiler with Nim HCR - if you want to understand how it was implemented: https://github.com/nim-lang/Nim/issues/8927
18:04:30disruptekright, so with no live compiler, you have to have the compiler at hand.
18:05:28Zevvdid zah implement this? and if so, is he not available to help out fixing?
18:05:42FromGitter<zacharycarter> he did not onqtam did
18:05:47disrupteki think he mentored it?
18:05:47Zevvah ok
18:06:39FromGitter<zacharycarter> I mean - I think the point is relatively moot - you can't hot reload code in a compiled language without having a compiler
18:06:44FromGitter<zacharycarter> I'm not saying that it's apples to apples
18:07:05FromGitter<zacharycarter> I'm saying that people that want to achieve a certain architecture have another option to achieve similar results with HCR
18:07:29Zevvtrue. I just ment that hcr is not plugins or scripting
18:07:46FromGitter<zacharycarter> I agree - it's just it's a means to an end
18:08:02FromGitter<zacharycarter> until we get new runtime I don't think the latter are solvable
18:08:11FromGitter<zacharycarter> without major work on the GC
18:08:56Zevvmakes sense. But clyybber noted hcr does not add too much complexity, and i guess he's a good authority on these matters
18:09:21Zevvso then it probably pays to keep it alive, if only for marketing purposes :)
18:09:35FromGitter<zacharycarter> well we should either kill it or fix it im
18:09:48clyybberbbl
18:09:50*clyybber quit (Quit: WeeChat 2.5)
18:10:04FromGitter<zacharycarter> having half baked features laying around doesn't do anyone any good
18:10:28Zevvis it generally broken, or only for specific cases? I only tried it once a few months ago
18:10:41FromGitter<zacharycarter> it works for simple toy examples - even the author admits that
18:10:48disruptekif it was robust, it'd be a foundation for a repl. but if not, well, then the repl is further out of reach.
18:11:15FromGitter<zacharycarter> but I started hot reloading code that called into code which used streams as well asserts
18:11:24FromGitter<zacharycarter> and I ran into all of these errors
18:11:32Zevvis it in the main docs/features or in experimental?
18:11:57disruptekhm, that's a great question.
18:12:09FromGitter<zacharycarter> https://nim-lang.org/docs/nimc.html#additional-features-hot-code-reloading
18:12:30Zevvif its not in experimental, maybe move it in there first. "half baked" is *bad* PR
18:12:34disruptekdoesn't look very experimental.
18:12:48*ljoonal quit (Quit: ljoonal.xyz)
18:13:43Zevvwe should make it then I guess
18:13:57FromGitter<zacharycarter> what I don't get is why the person who fulfilled the grant didn't finish it
18:14:10FromGitter<zacharycarter> when there's really no mention of just delivering a proof of concept
18:14:21FromGitter<zacharycarter> I assume he still got paid
18:14:46disruptekthe docs were probably part of the PR and weren't, uh, made more modest by the author.
18:14:48FromGitter<zacharycarter> maybe because it was deemed impossible?
18:15:04FromGitter<zacharycarter> and he only did what was possible?
18:15:26FromGitter<zacharycarter> but I don't really buy that since people implement hot reloading in C all the time
18:15:46FromGitter<zacharycarter> probably because it was a time constrained effort and he did what he could in the time allotted
18:15:56FromGitter<zacharycarter> and then it's just sat there since
18:16:47*ljoonal joined #nim
18:17:06*fredrik92 joined #nim
18:19:59shashlickhcr still depends on nimrtl which didn't do any better across dlls for me since i use threads
18:23:34FromGitter<alehander42> yes, afaik improving nimrtl is one of the main things, which is not really part of hcr
18:24:07FromGitter<zacharycarter> well that's been a thing for a while I believe
18:24:27FromGitter<zacharycarter> improving nimrtl - and it's not happened
18:24:41FromGitter<zacharycarter> there are issues about nimrtl and reloading dlls dating back several years
18:24:53FromGitter<alehander42> well yeah, but my point is, that most of the purely hcr stuff must be there
18:25:04FromGitter<alehander42> even if not stable enough
18:25:14FromGitter<zacharycarter> it could be
18:26:23FromGitter<zacharycarter> I mean it probably is since hcr depends on rtl
18:27:02FromGitter<zacharycarter> but I don't think the hcr failures I've run into are caused by rtl
18:27:35disruptekmaybe it wouldn't be too hard to run tests on the hcr using the existing testsuite.
18:29:47rayman22201I think Nimrtl hasn't been a priority for the same reason regions and the other GC's haven't been a priority. Araq thinks newruntime will supercede them.
18:31:50rayman22201At least that's my understanding. (Not to put words in his mouth)
18:36:53Zevvthere's a lot of things that will magically be fixed with the newruntime, if i undestand correctly :)
18:39:27rayman22201Lol. It may be too hyped. We'll see.
18:39:56*nif quit (Quit: ...)
18:40:05*nif joined #nim
18:40:12rayman22201At least it's not as bad as Vlang :-P
18:40:21Zevvits hard to get a feeling about how finished it is. nearing 90%? only half?
18:41:01rayman22201I think only Clyybber and Araq know the answer to that question.
18:41:33Zevvi hope so :)
18:44:09FromGitter<zacharycarter> Meh - Nim + Lua seems to be working well for my project so I don't think I'll try to deviate for now
18:45:02FromGitter<zacharycarter> and it's worlds better than C++ or C with Lua
18:45:37disruptekis it? why is that?
18:46:09FromGitter<zacharycarter> I don't have to manage memory
18:46:20FromGitter<zacharycarter> unless I want to
18:46:22disruptekoh, right.
18:46:58FromGitter<zacharycarter> also - the stdlib
18:49:40shashlickregions doesn't seem to release memory back to the OS even after deallocAll()
18:50:32Zevvwhat lies below regions? malloc or mmap?
18:50:33disruptekwhat's the status of quickjs on nim? is that wrapper shashlick wrote working well? i don't recall hearing much about it since it was created.
18:51:01disruptekshashlick: i think the recent issue explains the expected behavior best.
18:51:11rayman22201@shashlick: that's the bug that was fixed in shields PR. You need that PR
18:53:38rayman22201https://github.com/nim-lang/Nim/pull/11920
18:58:34shashlick@rayman22201 - the PR suggests it only fixes an issue with scratch region deallocation
18:59:07shashlick@disruptek - not sure if it was really used or not, i just created it
18:59:18shashlicklike the ultrajson wrapper a couple days ago
18:59:31shashlickhttps://pastebin.com/rPjGFHVD
19:00:55rayman22201iirc the PR fixed something to do with deallocAll as well
19:01:02FromDiscord_<treeform> I found it odd that there is no `nim --os:ios` is nim not supported on iOS?
19:01:26FromDiscord_<treeform> I also found it funny that `nim --os:android` makes it work on iOS (probably due to openGL ES)
19:01:40disruptektreeform: it's too hard to determine the platform difference between macos and ios.
19:02:31FromDiscord_<treeform> I tried `macos` and `macosx` but that did not compile right. Its funny that `android` is closer to iOS than `macos` is.
19:03:15rayman22201I just think nobody has tried to compile Nim on iOS before, so it never came up.
19:03:36shashlickmacos is old mac, macosx is newer mac
19:03:36disrupteknah, there's some `when ios` stuff in the codebase in a couple places.
19:03:38FromDiscord_<treeform> this guy did: https://github.com/yglukhov/nimx
19:03:59Zevvrayman22201: yes, but does it make sense to replace the threadpool given mratsims new Picasso initiative
19:04:37rayman22201@Zevv what context? Oh the GitHub comment!
19:05:33rayman22201Oh. Yes. It still makes sense. Mratsims RFC is very much a 2.0 long term goal. We really need a better threadpool for 1.0. short term solution.
19:06:06FromDiscord_<Shield> shashlick no it fixes withRegion too, basically all what those macros do is do a swap with tlRegion (the default region) then swap back, the problem that the swapping back was missing, MemRegion isn't a ref, so the deallocation does not happen correctly
19:06:10*nif quit (Quit: ...)
19:06:19*nif joined #nim
19:06:29Zevvrayman22201: make sense. but then agian, we can fix #2 on the old threadpool as well
19:06:46Zevvbecause replacing that will also be a project with tons of politics
19:06:47FromDiscord_<Shield> regions are basically memory pools, at least seq and strings are usable, once you go to gc:none, you are on your own to implement everything
19:07:59FromDiscord_<Shield> allocshared for regions is the same as allocation, I dunno how it'll work across threads
19:08:12rayman22201@Zevv: of course. I didn't mean to imply an order. We can fix #2 independently.
19:08:15FromDiscord_<Shield> I guess you have to use locks
19:08:37shashlick@Shield - even with the PR it doesn't really release mem back to the OS
19:09:29FromDiscord_<Shield> try with -d:useMalloc
19:09:51shashlickhttps://play.nim-lang.org/#ix=1RVv
19:09:53rayman22201@Zevv although, I think replacing threadpool will be much less politics. Especially since we have a good long term plan with mratsims RFC.
19:09:55shashlicki did and it didn't help
19:10:02FromDiscord_<Shield> I just did a manual fix for my nim folder, but the pr should be correct, it works on my end
19:10:49shashlickhow do i check actual mem usage
19:11:00Zevvrayman22201: but will it break current code, or is it 100% api compatible?
19:11:12FromDiscord_<Shield> I posted the procs in the end of the examples, same as normal gc
19:11:19Zevvif stuff will break, there will be politics
19:11:22FromDiscord_<Shield> and you're using withRegion, not withScratchRegion
19:11:32FromDiscord_<Shield> you didn't do any deallocation
19:11:40FromDiscord_<Shield> add m.deallocAll()
19:11:40shashlicki want to reuse the region from start to finish
19:11:54FromDiscord_<Shield> yeah, add that before or after sleep
19:11:56shashlickbasically using 1 region across gcs
19:12:12shashlickwhy isn't it reusing the mem from the vars that are no longer in scope
19:12:34FromDiscord_<Shield> like I said, it's manual memory management but with pages of memory instead
19:12:45FromDiscord_<Shield> you have to deallocate yourself
19:12:57shashlickso it's effectively like --gc:none
19:13:03shashlickkeep using ram until you dealloc
19:13:13shashlickdoesn't care that vars are no longer in scope and referred to
19:14:18shashlickso the `var istr` is used and thrown away so that mem should get returned to the region as unused
19:14:27FromDiscord_<Shield> yes, you use it if you want to isolate variables and ensure that they actually stay alive, you can have the dll have it's own region that gets freed when it's unloaded
19:14:32shashlickbut it just keeps using new memory, even in subsequent iterations
19:15:09shashlickmy use case is a long running text editor, the main exe and dlls are loaded until app exit
19:15:27shashlickthere's no unloading
19:16:21disruptekshashlick: are you using the patched version?
19:17:28shashlickyes
19:17:34shashlickdid you try my snippet
19:17:44disruptekwhere is it?
19:17:47shashlicksee RAM usage in procexp (Windows)
19:17:53shashlickhttps://play.nim-lang.org/#ix=1RVv
19:19:13shashlicki think even with regular GC, using a var in a for loop, Nim doesn't really clean up
19:19:31shashlickperhaps i add a GC_fullCollect
19:20:49shashlicknope, doesn't make a difference
19:21:15shashlickit's one thing not to return to the OS, it's another thing to use more and more RAM that should have been recovered from the previous for loops
19:21:17disruptekhmm.
19:22:13disruptekis the patch merged?
19:22:53disruptekbecause mine is growing, too. i thought the region would be deallocated outside its block
19:23:15disruptekmaybe defer m.dealloc... in the block?
19:23:22shashlickwell you need to call deallocAll() to deallocate a region
19:23:47shashlickbut i'm just asking for vars that go out of scope to release their mem back to Nim at least so it can be reused
19:24:05shashlickso in 3 for loops, it should use only 1 chunk of mem, not 3
19:24:16FromGitter<Varriount> Araq: https://stackoverflow.com/questions/10882277/properly-print-utf8-characters-in-windows-console
19:24:25FromGitter<Varriount> That may or may not help
19:25:17FromDiscord_<Shield> shashlick the patch is not merged
19:25:27FromDiscord_<Shield> you need to edit gcregions.nim yourself
19:25:32rayman22201@Zevv: Yuri's threadpool looks api compatible on the surface. I have not tested it in depth. There is probably some work to be done here.
19:26:04shashlicki did edit it
19:26:05FromDiscord_<Shield> also your i*i*i was causing an overflow, I stared at the screen for a good minute figuring out why that simple program crashed
19:26:16FromDiscord_<Shield> i x i x i
19:26:28FromDiscord_<Shield> also you're using echo 9999 times 3 times
19:27:10rayman22201@Zevv The good news is that I don't think threadpool is nearly as popular as async, so less people to complain about behavior changes :-P and we have the test suite + important packages to verify.
19:27:45shashlicki took the changes in withRegion()
19:28:39disruptekturning off echo (in unpatched nim) causes it to use less memory, too.
19:29:04shashlickit shouldn't use more ram in 2nd and 3rd loop
19:29:09FromDiscord_<Shield> https://play.nim-lang.org/#ix=1RVB
19:29:42AraqVarriount: I read that too, I'm close to giving up and will call 'printf'
19:30:01FromDiscord_<Shield> it frees memory correctly, but do note some stuff, echo, $ and any string concat causes garbage, it will pollute your current memory too
19:30:19shashlickwell, i don't want to deallocate until program exit
19:30:28Araqfor these the "scratch region" works well enough
19:31:15Zevvrayman22201: well, we could just make a pr pulling it in and see what happens. I lack the background knowledge to provide a rationale *why* we do that, though
19:31:18FromDiscord_<Shield> yes, you basically have to use multiple regions, the main one the should be perfectly controlled, and other ones you use to keep garbage out
19:31:41FromDiscord_<Shield> that*
19:31:45Zevvrayman22201: is "because araq said so" good enough?
19:31:47shashlickwell, point is that old unused memory isn't getting recovered
19:31:52shashlickso it is like --gc:none
19:32:31FromDiscord_<Shield> it is, but less harsher and you can use seq with it
19:32:49rayman22201@Zevv: I think so. we can also point to the irc logs for more context if needed.
19:33:47FromDiscord_<Shield> it also means that you should know what you're doing and how you're doing it, a text editor processing strings will generate garbage, any network handling generates garbage, there are a lot of string copy under the hood
19:34:08FromDiscord_<Shield> even a temp var can be troublesome
19:34:35FromDiscord_<Shield> OTOH, any bug is on you, you can't get any more personal with memory management
19:36:16rayman22201@Zevv I also know for a fact that Yuri's threadpool fixes some bugs that I have opened. This one for example: https://github.com/nim-lang/Nim/issues/11717
19:36:20shashlickya, regular GC reuses memory
19:36:34shashlickif the region doesn't reuse memory, it is pointless
19:36:38shashlickyou cannot use it for longer running stuff
19:37:04Zevvrayman22201: well, go for it I guess. one step at a time, someone should do it right?
19:37:38shashlicki'd have expected regions to effectively be the same algorithm as the refc gc
19:37:47rayman22201@Zevv It's on my to-do list :-)
19:37:57shashlickif a var went out of scope, you shouldn't have to wait until dealloc
19:38:01Zevvfirst let me finish my holidays :)
19:38:28rayman22201Lol. Yes! Go enjoy yourself!
19:38:54Zevvdont worry, doing just that
19:42:29shashlick@Araq - any reason why --gc:regions doesn't reuse memory of out of scope vars? forces it to be used only for short durations
19:42:41*Ven`` joined #nim
19:43:02shashlickin fact i wonder what the default region does - all that ram won't be released at all until program exit
19:43:35FromDiscord_<Shield> something has to give, gc doesn't reuse memory fast enough, I can max out my system memory with repeated memory allocations in loops
19:43:57shashlickbut even if you call GC_fullCollect doesn't do anything
19:43:58FromDiscord_<Shield> and as Araq said, if you want things to leave memory when they go out of scoop, you use withScratchRegion
19:44:18FromDiscord_<Shield> regions doesn't have gc_fullcollect
19:44:21shashlickwell you are still stuck with the main region which will do the same thing
19:44:39shashlickanything long running won't be able to use regions
19:45:13FromDiscord_<Shield> I don't think you understood the principle of regions
19:45:35FromDiscord_<Shield> you have a region for things that will last for the whole execution, then you make other regions for temporary stuff
19:45:58FromDiscord_<Shield> and use withScratchRegion for quick and dirty
19:46:35FromDiscord_<Shield> you won't run out of memory if you know where you're allocating
19:47:47shashlicki get the scratch regions, but requiring scratch for every random temporary vars is excessive
19:48:05shashlickmy snippet is a bad example
19:50:21shashlicki get the idea but not providing a way to clean up (automated or manual) without full destruction makes it very limited
19:50:48FromDiscord_<Shield> it is no different than using manual allocation, but less ugly, but I see what you're getting at, the problem is regions cannot handle holes, it will keep allocating memory at the end until you purge the whole region
19:53:13shashlicki'd have thought it would be easier to use the standard gc behavior but just have multiple "spaces" to work within and throw out
19:54:18shashlicki cannot think of any use case where regions will not prove to be a big pain
19:54:42shashlickanyway, it doesn't solve my problem so back to the drawing board
19:57:27rayman22201At the risk of being annoying, may I suggest newruntime?
19:58:54shashlickagain, need some examples to look at 😄
20:01:31FromDiscord_<Shield> I'm looking at the tests
20:02:44*krux02 joined #nim
20:02:46*rockcavera quit (Remote host closed the connection)
20:15:45AraqShield: regions are inherently less flexible than a real heap is
20:15:51Araqand that matters for UI apps
20:27:47FromDiscord_<Shield> I see, it looked like the better option instead of using gc:none, using minimal nim is counter productive
20:32:34FromGitter<deech> The `c2nim` docs are down: https://nim-lang.org/docs/c2nim.html.
20:33:38dom96hello everyone, how's everyone doing?
20:35:12Zevvshould we *all* answer?
20:37:29Araqdeech: https://nim-lang.org/docs/tools.html doesn't link to that anymore
20:37:57dom96Zevv, why not? :P
20:38:57shashlicki feel like a rat stuck in a cage running around in circles with nowhere to go
20:39:41dom96ahh, so not great. Should I be concerned for your well being?
20:41:47Araqshashlick: I've looked at your DLL bug report
20:42:14Araqbut you write "it works if put in a main proc" so that cannot be the real problem, can it?
20:42:27Araqputting stuff in a main proc is a valid workaround
20:49:07shashlickno i'm cool, just frustrated
20:49:25Araqshashlick: saw my question?
20:50:31*narimiran quit (Remote host closed the connection)
20:50:51shashlickya, but that's just a minimal example - i run into the issue in my code which isn't isMainModule
20:51:12shashlickfurther, i worked around it by changing the HashSet to a seq
20:51:23shashlicki run into a similar assert in another way though
20:52:27shashlickbut i suspect that is because of multi-gc
20:53:55Araqshashlick: hmm ok, so you think your minimal example is also what makes it fail for feud?
20:56:56shashlicki suspect there's something funky going on with dll loading causing asserts
20:57:22shashlickif you know it is because of multiple gcs, then i can only use newruntime or alloc0()
20:57:52shashlickalloc0 will be a pain to use to interop between main exe and dll
20:58:11*Trustable quit (Remote host closed the connection)
21:00:06shashlickhere's my data structure - https://github.com/genotrance/feud/blob/master/src/globals.nim#L14
21:00:07*theelous3 joined #nim
21:00:20shashlicki use cindex for the plugin dll to tell me what procs are exported
21:00:35shashlicki get that info in onLoad() and then symAddr() all of them
21:00:44*Kaivo quit (Quit: WeeChat 2.5)
21:00:46shashlickthey are stored into callbacks
21:01:29shashlickso what's happening is that cindex comes from the dll and stored into Plugin which is allocated in the main exe
21:01:43shashlicki see most mixups in the callbacks table
21:01:55shashlickstuff disappears or random stuff shows up in it
21:04:46shashlickin some sense, i'm trying stuff that isn't supported/tested
21:04:53shashlicki am exchanging data across exe and dll while using threads so nimrtl doesn't work
21:05:22shashlickboehm used to work in 0.19.6 but doesn't anymore, so we could debug that but it seems like a distraction from newruntime
21:06:07shashlickI think https://github.com/nim-lang/Nim/issues/11914 might be a better use of your time
21:14:07FromDiscord_<Shield> https://github.com/nim-lang/Nim/issues/11914
21:14:22FromDiscord_<Shield> oh I posted before finishing reading
21:14:33FromDiscord_<Shield> I wonder what is wrong in hashsets
21:14:56FromDiscord_<Shield> can somebody explain how memory works between the main exe and dlls?
21:19:55FromGitter<Varriount> What do you mean?
21:23:23*nif quit (Quit: ...)
21:23:33*nif joined #nim
21:27:54FromDiscord_<Shield> I don't understand why there should be one gc instance per address space, why wouldn't gcref/unref help with memory across dlls?
21:28:37rayman22201A dll loads into the main exes memory space just like C. But the problem is that for Nim the dll doesn't know about the main exes gc, and may try to init it's own GC. This causes all kinds of problems. Nimrtl is supposed to solve this by having a common GC api that everything uses.
21:29:33shashlicksee my link further up to globals.nim and how cindex and callbacks are working
21:29:50shashlicksince it is two GCs, they cause issues
21:31:36rayman22201Both GCs assume they are the only GC and that they control the entire heap. So they interfere with each other. The GC algorithms aren't designed to compose with multiple instances running.
21:32:58*solitudesf quit (Ping timeout: 245 seconds)
21:33:39shashlickis the heap at the same spot in memory or do the two GCs look at different mem locations
21:34:04shashlickif it is the same then even if I don't copy data back and forth and use alloc, i might have the GCs still mix up unshared data
21:34:07shashlickand cause other issues
21:38:42rayman22201https://github.com/nim-lang/Nim/blob/devel/lib/system/gc.nim#L99
21:38:56rayman22201probably two different locations based on this, but they will run into each other eventually
21:42:58shashlickso using alloc() to interact between main exe and dll isn't fool proof either
21:43:05shashlickwith multiple GCs, i'm asking for trouble
21:43:39FromDiscord_<Shield> well, Araq did suggest using normal gc without nimrtl and see how it goes
21:43:53AraqI didn't
21:44:12Araqyou need the GC to be in nimrtl so that every DLL can share the heap
21:44:37Araqthat's not a problem unique to GCs, but to every allocator, C has the same problem, it's just hidden better
21:51:02*Ven`` quit (Quit: My MacBook has gone to sleep. ZZZzzz…)
22:00:20shashlickso with nimrtl, I get this - http://ix.io/1RWl
22:01:56FromDiscord_<Shield> why so many threads?
22:02:57*fredrik92 quit (Ping timeout: 245 seconds)
22:03:13shashlickthere should just be 2, i don't know why there's so many cause it hasn't loaded anything
22:03:20shashlickmain thread and monitor thread that loads dlls
22:07:44shashlickso in summary, cannot use nimrtl (threads), regions (long running), alloc (still multiple GCs which could cause other havoc)
22:08:21shashlicki can continue to debug with boehm
22:08:38shashlicknewruntime also doesn't seem practical since i use tables a lot
22:09:21shashlicki have low confidence i'll make any progress with boehm since i've already spent several weeks on that
22:09:55disruptekfeelsbadman
22:12:56rayman22201You might be able to debug nimrtl. Based on the stack trace you posted, It looks like it gets a bad pointer here : https://github.com/nim-lang/Nim/blob/devel/lib/system/threads.nim#L165
22:14:40rayman22201Wrong version. This link: https://github.com/nim-lang/Nim/blob/v0.20.2/lib/system/threads.nim#L165
22:16:18FromDiscord_<Shield> did you try gc:go?
22:16:23FromDiscord_<Shield> why wouldn't alloc work?
22:16:42FromDiscord_<Shield> with pointers I mean, not refs
22:17:05shashlicki rebuilt nimrtl.dll with debug info to get a better stack trace
22:17:12shashlickand now it loaded!
22:17:24shashlickeverything else was built with debug, perhaps that makes a difference?
22:18:10shashlickgosh was it just that?
22:18:29FromDiscord_<Shield> in the docs it tells you that using nimrtl with threads:on is untested, but then again it's compiled with that flag by default
22:19:12FromDiscord_<Shield> try everything with release build and see
22:23:09shashlickcool, anyway, need to test everything, will report back
22:24:41FromGitter<deech> Araq, thanks!
22:26:19FromGitter<deech> Is there a way to generate entire modules? I want to sequester a bunch of generated code into a named module not have it pollute the current namespace with `{.dirty.}`.
22:39:16*actuallybatman quit (Ping timeout: 246 seconds)
22:41:08FromDiscord_<Shield> I still don't fully understand "owned", does that mean it's guaranteed to be consumed and owned by one owner?
22:41:17FromDiscord_<Shield> what happen if you just discard it?
22:54:50shashlicknope - release mode crashes just the same (release feud + release nimrtl crashes), (debug feud + release nimrtl crashes), (debug feud + debug nimrtl worked but doesn't anymore!?)
22:57:06shashlickhere's a better stack trace http://ix.io/1RWB
23:10:24*krux02_ joined #nim
23:11:02rayman22201That stack trace is interesting. How does it get from gc_common.nim:284 to system/threads.nim:152? That makes no sense. gc_common.nim:284 is just a null check? https://github.com/nim-lang/Nim/blob/v0.20.2/lib/system/gc_common.nim#L284
23:11:09shashlickmakes no sense ya
23:11:28shashlickwell the bigger thing is line 272
23:11:41shashlicki am setting -d:useNimRtl - why is it still going in there
23:12:18rayman22201After that, threadProcWrapStackFrame is calling threadProcWrapperBody, which calls threadProcWrapStackFrame again. So it looks like it is trying to wrap the thread stackFrame twice.
23:13:16*krux02 quit (Ping timeout: 264 seconds)
23:13:55rayman22201ohhhh. I see what you mean about 272. interesting. Stupid question, but are you setting -d:useNimRtl when building both Feud and nimrtl?
23:14:06shashlicki am when building feud and all dlls
23:14:17shashlickbut you cannot set -d:useNimRtl for building nimrtl due to this
23:14:25shashlickhttps://github.com/nim-lang/Nim/blob/v0.20.2/lib/system/inclrtl.nim#L22
23:14:45shashlickmaybe line 152 is wrong and should be commented out? or changed to something else
23:14:49rayman22201ahhh. That makes sense lol
23:15:29shashlickline 152 in threads.nim that is
23:15:45rayman22201Well, nimGC_setStackBottom must exist
23:15:51rayman22201it's an important proc
23:16:22*clyybber joined #nim
23:17:59rayman22201It's like this stack trace is out of order. It's kind of bizarre
23:18:20shashlickwell, so i think nimrtl.dll is running that code
23:18:40shashlickand it is weird that docs say threads are not supported or tested but the .cfg has --threads:on
23:19:59rayman22201ah. duh. gch.stack.bottom is the segfault.
23:20:23rayman22201ok, looks like you have gdb. can you print the value of `gch` at the time of the crash?
23:23:41rayman22201what does `{.rtlThreadVar.}` do? I assume it's some special version of `{.threadVar.}`
23:25:53shashlickugh no symbol in current context
23:25:55shashlickfor gch
23:26:36FromDiscord_<Shield> rtlThreadVar means threadVar when thread support is on and means nothing when it's off
23:26:47shashlick{.pragma: rtlThreadVar, threadvar.}
23:26:57rayman22201my hunch is that `gch` is a thread local variable, and you are starting a new thread, so `gch` doesn't exist for that thread
23:27:39rayman22201@Shield thanks!
23:28:22*stefanos82 quit (Quit: Quitting for now...)
23:28:37rayman22201I wonder if the bug is this stupid. Try switching lines 152 and 153 in threads.nim
23:28:40rayman22201https://github.com/nim-lang/Nim/blob/v0.20.2/lib/system/threads.nim#L152
23:30:16rayman22201I actually don't understand how it makes sense to call `nimGC_setStackBottom` on a `gch` that may not exist yet
23:30:35shashlicksee line 182 in gc_common
23:30:37shashlicksame order
23:32:03rayman22201weird
23:32:22rayman22201well, somehow the `gch` struct is getting borked.
23:32:28*vesper11 quit (Ping timeout: 246 seconds)
23:33:20*ng0 quit (Ping timeout: 260 seconds)
23:34:01rayman22201bah. I have to go. bbl
23:34:33shashlickno problem thanks for taking a look
23:38:07clyybbergn8
23:38:08*clyybber quit (Quit: WeeChat 2.5)
23:43:06*ng0 joined #nim
23:47:10FromDiscord_<Shield> did that solve it? what's about bohem that makes it work?
23:47:45*ng0 quit (Quit: Alexa, when is the end of world?)
23:48:32shashlicknope i'm still stuck - trying nimrtl and have crash issues due to threads
23:48:57shashlickboehm is aware of cross thread data so no multi-gc issues
23:49:09shashlickbut it no longer works on 0.20.x for some reason
23:49:31shashlickand there's no real support for boehm in the community
23:51:52FromDiscord_<Shield> can you test go?
23:52:28*abm joined #nim
23:52:49shashlickhow will it help?
23:53:12shashlickand what do i need to get it running besides --gc:go
23:57:01FromDiscord_<Shield> you grab the go dll just like bohem
23:57:31FromDiscord_<Shield> I'm curious if it'll have less bugs
23:57:56shashlickI'm building
23:58:03shashlickBut still searching the dll
23:58:08shashlickNeed 64 bit