<<10-02-2013>>

00:02:48mwcampbellAt the risk of stirring up controversy, I suggest a radical reduction in the size and scope of the Nimrod standard library. I think the standard library should be just large enough to support the compiler itself and the Babel package system.
00:03:20mwcampbellWe can see some of the problems of an all-inclusive standard library by looking at some complaints about Python's: http://lucumr.pocoo.org/2009/3/2/std-stands-for-sleazy-tattered-and-dead/
00:05:16mwcampbellBreaking out wrappers and convenience libraries into their own packages would simplify Nimrod's release cycle. Could help get it to 1.0 faster
00:07:29Araqwhat you say has some merit
00:07:53Araqbut the Babel package system is not up for it yet
00:08:09Araqand we constantly fight over how to do it properly
00:08:34Araqthe fact that I never found a package manager that really works doesn't help ;-)
00:09:53Araqand it really depends on who maintains what; there is no productivity gain if it ends up that the core devs need to switch between different git repositories more often
00:16:23*XAMPP joined #nimrod
00:21:45Araqbtw I noticed Python's stdlib has its bad parts but it's been very good for marketing python
00:22:02Araq"it comes with batteries included!"
00:26:40mwcampbellMarketing. That thing that stands in the way of developers being honest with their users about trade-offs, even when those users are other developers.
00:29:40Araqiirc dom96 is happy to let somebody take over Babel. Interested?
00:30:01mwcampbellWhat areas need work?
00:30:27Araqfor a start it cannot even download dependencies
00:30:46Araqnor do packages specify dependencies
00:31:50Araqand there is some annoying duplication with .babel files and the information that's kept in the global list of registered projects
00:32:01Araqand the global list is a single JSON file
00:32:31Araqwhich I despise but then you can't upload sqlite files to github
00:32:48Araqas that would be a binary file in a git repository
00:33:08mwcampbellWhat's wrong with using JSON?
00:33:08Araqand then all the git nazis show up to tell you how clueless you are ...
00:34:00AraqI'd use a directory of babel files or an sqlite database instead of an ever growing JSON file
00:34:41mwcampbella directory of babel files seems quite sensible to me
00:35:01Araqyeah but then you have to version these
00:35:34Araqas the dependencies/requirements/build instructions can change over time
00:36:09Araqit's an ever growing pile of complexity
00:36:24Araqand in the end it breaks and you have to learn the system to fix it
00:37:01*Araq is no fan of package management
00:38:16mwcampbellHave you considered throwing all of that out and just using Git submodules instead?
00:38:32mwcampbelli.e. each project repo has submodules for all the things it depends on
00:38:43Araqbut then everybody has to use git
00:38:53mwcampbellNot necessarily.
00:38:59Araqplus I have some experience with git submodules
00:39:03mwcampbellRecent versions of Mercurial can have Git subrepos
00:40:07Araqmy linux mint says "everything is up to date"
00:40:24Araqso I click to see the list of available updates
00:40:33Araqand it's not empty :P
00:41:35Araqand no, I don't like git submodules
00:41:42Araqthey never worked for me
01:06:46Araqgood night
01:07:48mwcampbellgood night
01:08:51*mwcampbell quit (Quit: Leaving)
01:13:40*SchalaZeal joined #nimrod
01:14:20SchalaZealI know when defining types Tfoo|TBar works, but does it work for field/var declarations?
01:22:22*Anaphaxeton quit (Ping timeout: 244 seconds)
01:24:03*Anaphaxeton joined #nimrod
01:27:47AraqSchalaZeal: no it doesn't
01:27:59Araqit's nothing more than an implicit generic type
01:28:07Araqbut I'm sleeping ;-)
01:28:49SchalaZealah ok
02:00:07*fowl joined #nimrod
02:25:24*fowl quit (Ping timeout: 248 seconds)
02:39:25*q66 quit (Quit: Quit)
03:48:01*XAMPP quit (Read error: Connection reset by peer)
04:30:38*SchalaZeal quit (Quit: Konversation terminated!)
05:59:01reactormonkAraq, hehe, just seen an ipad app which had nimrod as choice for code
05:59:12reactormonkAraq, 'code2go' - among ~ 30 other
05:59:51reactormonkAraq, btw, ever tried the tk text widget?
06:08:12reactormonkhttps://github.com/pcwalton/sprocketnes based on rust...
07:56:22*FreeArtMan joined #nimrod
08:57:20*gour joined #nimrod
09:08:22gourAraq: this does not sound good: "IUP does not have a wide localization feature, it only includes support for messages in English and Portuguese. And it does not have support or Unicode characters." along with "Why Not Mac?"
09:23:38shevylet's implement nimrod on parrot VM!!!
09:30:12gourto revive parrot? excellent idea :-P
09:31:29shevy:D
09:31:43shevywell I guess it is too dead now to interest anyone
09:31:50shevybut imagine 3 years ago! set the sails on that ocean!!!
09:33:09Araqparrot's GC is unimpressive
09:33:31Araqmake that: let's implement Parrot in Nimrod ;-)
09:52:18*zahary1 joined #nimrod
09:53:45Araqping zahary1, zahary
09:54:15zahary1hi
09:54:24Araqbad news ...
09:54:53AraqI enabled the cycle collector and --gc:v2 -d:release and got a segfault :-/
09:55:44Araqmost of the time it works though
09:55:53zahary1does it look like similar to the gradha's bug report for aporia?
09:56:33AraqI don't know, it crashed quite late
09:57:07Araqmaybe it was osproc related
09:57:55Araqanyway here are more results:
09:58:11zahary1well, there is one test case that fails with something that looks like premature freeing. I should investigate that
09:58:17Araq60259 lines compiled; 3.446 sec total; 139.142MB -- v2
09:58:19Araq59818 lines compiled; 3.377 sec total; 141.144MB -- old GC with repaired cycle collector
09:58:46zahary1how extensive is the repair?
09:59:30Araqwell I figured 'excl' is allowed while traversing a TCellSet
10:00:28Araqbut it's still a large diff
10:00:41zahary1but it still does pruning like before? it looks whether the next object to traverse is in the cycleRoots sets?
10:01:16Araqwell I've implemented the coloring of Bacon
10:01:25Araqno pruning
10:01:42Araqbut it can free immediately, no need to go through the ZCT
10:02:19Araqhowever I'm still too stupid to figure this out:
10:02:46Araqnew n; n.next = n; GC_collect(); # 'n' is on the stack
10:03:13Araqbut it's cyclic and incref doesn't add it to the cycle roots
10:03:42Araqyou can't assume that the stack really references n
10:04:46zahary1new objects should be considered cycle candidates
10:05:00Araqbut where does the paper say that?
10:05:20zahary1it's explained better in one of the next papers by PAz
10:05:28AraqI read the relevant parts 3 times ...
10:05:51Araqand that's a big wtf
10:06:11Araqbecause then you basically traverse each fresh graph once
10:07:53Araqand it's still wrong then:
10:08:02Araqif s.color == rcPurple and s.refCount > 0:
10:08:09Araqis it in 'markRoots'
10:08:19zahary1yes, they propose couple of additional measures that try to postpone the traversal of new objects until they are either freed, because of RC=0 or removed as condidates because of incref
10:08:19Araqbut new objects start 'black' ...
10:09:19zahary1color = rcPurpe ans s.refCount > 0 is designed to deal with dead object (that became dead because of RC = 0)
10:09:35Araqbut it's wrong
10:09:41zahary1why so?
10:10:13Araqnew n; n.next = n;
10:10:28AraqincRef sets 'n' to black
10:11:38zahary1aha, I see. let me think about it
10:13:07Araqthis whole "it's only a cycle when decRef produces an RC > 0" just doesn't work when you don't RC the stack
10:13:36zahary1well, I don't have the special incref treatment right now
10:15:51zahary1it could still be fixed with the hasStackRef bit that I don't use yet
10:16:10zahary1I have left some notes about it in the code
10:17:35zahary1# this is safe only if we can reliably track the fact that the object
10:17:36zahary1 # has stack references. This could be easily done by adding another bit
10:17:36zahary1 # to the refcount field and setting it up in unmarkStackAndRegisters.
10:17:36zahary1 # The bit must also be set for new objects that are not rc1 and it must be
10:17:36zahary1 # examined in the decref loop in collectCycles.
10:17:36zahary1
10:17:43zahary1 IncRefRemovesCandidates = false
10:20:51Araqperhaps it works because 'n' still has to be kept in the ZCT
10:21:07Araqand then you need a full decRef in the ZCT handling
10:21:16zahary1well, do you have some objections to the fix above?
10:21:37Araqyes, the stack is not sufficient
10:21:46zahary1why so?
10:21:59Araqbecause it may not contain any 'n'
10:22:20Araqnew n; n.next = n; performLotsOfWork(); collect()
10:22:27zahary1that's not the point. what you need is to know that it contained a stack ref in the previous collection
10:22:48Araqthere is no previous collection
10:22:51zahary1that way you can detect that the stack ref was cut off
10:23:15zahary1yes, but you can assume new RC0 objects to have a stack ref in the previous collection
10:23:35Araqhow so?
10:23:57zahary1    irc://chat.eu.freenode.net:6667/# The bit must also be set for new objects that are not rc1 and it must be
10:23:57zahary1    irc://chat.eu.freenode.net:6667/# examined in the decref loop in collectCycles.
10:24:49Araqso that 'new' adds it to the cycle candidates
10:25:09Araq*so you still keep that 'new' adds to cycle candidates?
10:25:17zahary1yes
10:25:21Araqwell sure then it can work
10:25:29Araqbut I'm trying hard to avoid that
10:25:42zahary1the question is can you also have "addref removes candidates"
10:26:21Araqthe question is: do we need to do it in 'decRef' if we add it to 'incRef'?
10:27:30zahary1I have a switch for that. it doesn't change much
10:31:19Araqyeah but I'm thinking about correctness
10:31:34Araqnot what it means to the performance of bootstrapping
10:31:55Araq(which is faster with --gc:markAndSweep anyway)
10:32:25Araq(and is the way to go for caas I think)
10:32:36zahary1you can do add cycle in incref - you must also add new objects to the ZCT then
10:36:08Araqwell new objects are added to the ZCT anyway
10:36:33zahary1yep
10:36:40Araqbut I guess my ZCT replacement is wrong then
10:36:45zahary1hmm, the new GC doesn't free all the memory after reset
10:37:23Araqdunno what you measure, there is always fragmentation
10:37:38Araqand the memory allocator does not free everything
10:37:40zahary1stays at 308MB, I'll investigate
10:38:01Araqthe cycle roots array is never really freed
10:38:14Araqit's reused for the next runs though
10:40:01zahary1I meant the mark and sweep GC
10:40:10Araqugh
10:40:43Araqwell ... perhaps the codegen for root marking is wrong
10:42:59Araqdoes it grow? or is it bounded at 308MB?
10:44:07zahary1there is a codegen error on recompilation, still looking at it
10:50:55Araqyay ... :-/
10:53:23*gour quit (Disconnected by services)
10:53:30*gour_ joined #nimrod
11:05:07zahary1the memory grows, but at much lower rate than the old GC (20MB on a recompilation that used to bring 100MB)
11:05:58*gour_ is now known as gour
11:10:28Araqwhat's the codegen error?
11:12:32zahary1it wasn't some kind of corruption. there was a problem in the code merging logic that wasn't triggered before for yet unknown reason
11:12:51Araqah ok
11:14:02Araqbtw I wanted to ask you about your changes to setLengthSeq
11:15:16Araqwhy do you do the 'len0 = gch.tempStack.len' stuff? isn't tempStack guraranteed to be empty?
11:15:18*q66 joined #nimrod
11:17:09zahary1in this particular function, yes, but not in some other places. it was just a precaution, and planned to convert the code to a bit higher level templates at some point
11:37:47Araqalright
11:38:02Araqgc_ms still deallocs in growObj btw
11:38:26AraqI still don't know what part in the compiler accesses an old sequence
12:01:48zahary1I never got a crash with retired buffers in the complier. only thought experiments crashed with that one :)
12:05:36zahary1the new GC is remarkably simple and it can probably easily become even faster. there isn't a leak after all I think.
12:05:59zahary1it was just some conservatively marked objects, me misreading the GC stats output and severe memory fragmentation (which affects the previous GCs too and is something that I wanted to talk to you about)
12:09:57Araqit's hard to make it faster by playing with the algorithm
12:10:37Araqit can easily be made faster if we optimize the allocator's data structures to it
12:12:04Araqlike described for the new GC of Luajit
12:12:32Araqwhat causes the severe fragmentation?
12:15:07Araqbtw it's no wonder MinimumStackMarking doesn't work ;-)
12:15:27Araqyou set the stack top within 'newObj' ... that can't work
12:18:33zahary1in some of the papers it was mentioned that our current way of assigning pages to cell sizes is known for bad fragmentation.
12:19:09zahary1it could be some other bug causing the high memory usage tho. I just disabled marking altogether and 7 remaining objects still keep the memory at 276MB
12:19:58zahary1why is newObj a bad "border" for the user stack?
12:21:00AraqnewObj; smashStack; useLargeStack; let myTemp = myGlobal; myGlobal = nil; GC_collect()
12:22:45zahary1so? GC_collect sets the stack top too
12:23:13Araqoh I missed that; why do you do it newObj then?
12:23:20Araq*it in
12:24:26zahary1because, newObj can call collect too (not the public FullCollect proc, but the internal one)
12:24:33zahary1I set the top only at the public procs
12:24:55Araqah
12:49:24Araqzahary1: I played a bit with merging the m&s collector into the RC-based
12:49:59Araqand also to enhance the m&s one with a write barrier so that only deltas are re-marked
12:50:47Araqbut it's harder than it looks and the compiler simply does need ever more memory as we can't free proc bodies etc.
12:51:23Araqso I fail to see how introducing generations actually improve anything:
12:52:45Araq1) any kind of write barrier is expensive as the C compiler doesn't know about its semantics and most write barriers touch some global state in an 'if'
12:53:06Araqwhich should prevent some optimizations
12:54:29Araq2) I don't think the compiler produces enough shortly lived objects so that a generation0 GC run would pay off
13:01:33zahary1I'm more interested in a write barrier for incremental GC. it's through that with automatic stack objects the generational hypothesis can be questioned
13:01:58zahary1* it's true that *
13:02:18Araqwell we have an incremental GC
13:02:41zahary1which one?
13:02:48Araqthe old RC-based one?
13:03:01zahary1that doesn't collect objects :)
13:03:06Araqhu?
13:03:48zahary1well, the correct collectCycles implementation is not incremental
13:04:13Araqtrue, but I don't think this can really be solved in a satisfactory manner
13:04:30Araqthat's why I exposed it for the realtime GC
13:05:28Araqyou can make the cycle collector incremental though
13:05:38zahary1that's why it may be worth trying out a tracing incremental collector too
13:06:26Araqtracing inherently requires to mark full graphs before you can free anything
13:06:39Araqok, you can migitate that with generations
13:06:40zahary1otherwise, the generation GCs seem to also be about fast allocation path for the new object - i.e. the bump-pointer nurseries that are freed at once in the simple collections
13:07:17zahary1but that requires precise stack marking
13:07:52AraqI really want that "it's not touched what hasn't changed" property
13:08:12Araqhaving lots of mostly static readonly data is common
13:08:58zahary1yes, maybe the cycle collector will perform better in most projects, which are not so pathologically interconnected as the symbols in the compiler
13:09:21Araqthinking about it, m&s is also really stupid for that; marking will take more and more time
13:09:43Araqas the compiler allocates objects that just stay alive
13:09:47zahary1yep
13:10:32Araqhrm, looks like quadratic behaviour
13:11:15Araqin fact, I always wondered if GCs based on marking don't mess with the big O
13:12:57zahary1you can have a "marked" bit in the TCell
13:13:28zahary1that will get you a more linear behavior, but still linear to the world size is not that good
13:13:54Araqin gc_ms you have a full word as the refcount field is not used
13:13:58Araqtry to make use of it
13:14:15Araqyou'll see it's worse than the TCellSet approach
13:14:25zahary1it's worse?
13:14:45Araqyes
13:14:50Araqin theory and in practice
13:15:06Araqthough admittedly I didn't spent too much time playing with it in practice
13:15:08zahary1what the theory? less writes, less invalidates cache lines?
13:17:23Araq for c in gch.allocated.elementsExcept(gch.marked): free(c)
13:17:44Araqso what we touch here are the objects that should be freed
13:18:05Araqwith the bit within the object you'll touch *every* object instead
13:18:43Araqplus you have the problem that you modified the live object's header
13:18:54Araqand you either need to swap the bit's semantics
13:19:13Araqor set it back to 'unmarked' again for the next GC run
13:19:17zahary1but you still touch memory for every object in the bit mask in TCellSet
13:19:41Araqbut the TCellSet tries hard to compress the information
13:20:00Araqand it doesn't touch the TCell itself
13:20:07zahary1so less invalidated cache lines as I said
13:20:28Araqwell yeah, but it also affects paging etc.
13:20:39Araqso I simply speak of "touching"
13:22:15Araqit is comparable to what luajit's new collector will do
13:22:42Araqexcept that the bits are kept at the block start whereas I use a hash table of bitsets instead
13:23:12Araqwhich should be slower, but TCellSet is what I had lying around ;-)
13:25:00zahary1yeah, maybe we should implement Mike's GC before him :) only finalizers don't fit well into the bitwise sweeping scheme
13:25:30zahary1the mark and sweep GC doesn't honour the order of finalizers yet btw
13:25:45AraqI don't think it should
13:26:15Araqwe have destructors in the language for that
13:29:17Araqoh lol
13:29:33Araqer ... no
13:31:51zahary1we can postulate that the finalizer should not touch other heap objects, but that's some drop of guarantees
13:33:07Araqbut the only use-cases I have in mind do fullfill this postulation
13:33:33Araqaka "close a file in a finalizer"
13:33:51Araqwhich is bad practice but common
13:34:34Araqweak references don't need more guarantees either
13:34:46Araqor do they?
13:37:50*gour is doing some light pascal tutorial reading..seeing some influences on nimrod (which may come directly from Ada as well), but wonder what are some of the things missing in Pascal which led Araq to design Nimrod
13:40:10Araqgour: pascal has no macro system, no GC, a broken object model and a overly verbose syntax
13:40:28Araqoh it has no generics either
13:40:47Araqnowadays it has generics though
13:41:19gourfpc's object model is included in "broken one" ?
13:41:48Araqreference based semantics without a GC and without implicitly invoked destructors?
13:42:14Araqthat surely counts as "broken"
13:42:47gourok...fair-enough. thanks for the input
13:45:23Araqit doesn't matter that much UIs though and the *tools* Delphi/Lazarus are really nice
13:45:30Araq*for UIs
13:49:11gourwell, what if you have need for 'nice UI' ?
13:50:26AraqI try to avoid UIs ;-)
13:50:30*gour is not happy seeing that today many count web as the only UI :-(
13:51:05gouri'd also like to do, focusing on writing custom libs...but there is need to put those functions in real use
13:51:41Araqyou can try to use Gtk, Glade and Nimrod
13:52:48gouryeah...i did my 'divination' today asking about nimrod/fpc(lazarus)/ada/ocaml and have to go deeper into intrepretation...will do it later
13:56:38*zahary1 quit (Read error: Connection reset by peer)
13:56:46*zahary1 joined #nimrod
14:14:10NimBotAraq/Nimrod a88394f Araq [+0 ±1 -0]: working cycle collector for old GC
14:14:10NimBotAraq/Nimrod e97d3f4 Araq [+0 ±5 -0]: open arrays documented in the manual
14:14:10NimBotAraq/Nimrod 6c9c266 Araq [+0 ±3 -0]: Merge branch 'master' of github.com:Araq/Nimrod
14:14:10NimBotAraq/Nimrod a479ae9 Araq [+0 ±1 -0]: updated news.txt
14:17:24*zahary1 quit (Ping timeout: 248 seconds)
14:21:41gourwhen will forward decl. disappear?
14:28:22dom96zahary is working on that
14:29:41*zahary1 joined #nimrod
14:33:59gourwho is 'officially' the 3rd core dev?
14:34:35dom96me I guess
14:34:44gourcongrats!!
14:36:22gouris there some help in using gtk's object introspection to create bindings for nimrod, e.g. for gtk3?
14:36:32gours/some help/some use
14:37:22dom96no idea.
14:38:03gourmany projects use those to almost automate creating of bindings
14:38:26*zahary1 quit (Ping timeout: 252 seconds)
14:38:45dom96I would just use c2nim
14:42:03*zahary1 joined #nimrod
15:08:04*XAMPP joined #nimrod
15:08:04*XAMPP quit (Changing host)
15:08:04*XAMPP joined #nimrod
15:13:23*XAMPP quit (Ping timeout: 256 seconds)
15:42:51gourAraq: have you seen this opportunity: http://forum.dlang.org/thread/[email protected]?page=1 ? andm, btw, i'm curious what are your main objections against D? (not looking for D bashing)
15:45:30gourtutorial1 says: "Documentation comments start with ##." but i do not recall more info about them?
15:47:57gourthis is bug: 1.0e9 (one million)...iirc, it should be 1.0e6 :-)
15:56:08Araqgour: thanks, good catch
16:12:04*Anaphaxeton quit (Remote host closed the connection)
16:15:50*Anaphaxeton joined #nimrod
16:16:20*zahary1 quit (Ping timeout: 252 seconds)
16:21:47*zahary1 joined #nimrod
17:41:11*fowl joined #nimrod
19:17:39*shevy quit (Ping timeout: 260 seconds)
19:29:13*shevy joined #nimrod
19:38:36gourdom96: which distro you run?
19:41:00dom96arch
19:46:56*zahary1 quit (Quit: Leaving.)
20:02:59*shevy left #nimrod ("I'll be back ... maybe")
20:09:09gouri'm on debian and there is only glade-3.12 which, iirc, does not support gtk2, right?
20:57:37*XAMPP joined #nimrod
20:57:37*XAMPP quit (Changing host)
20:57:37*XAMPP joined #nimrod
21:04:13*FreeArtMan quit (Quit: Leaving)
21:19:08*gradha joined #nimrod
21:28:12reactormonkdom96, should nimrod-mode go to nimrod-code?
21:29:48dom96not necessarily. Would you like it to belong to this tiny github "organisation"?
21:31:21reactormonkdom96, is it useful to go there?
21:31:35dom96not really.
21:32:10gourdom96: you don't use glade or use old one?
21:32:21dom96gour: I don't use glade at all.
21:32:48gourdom96: but there is support for it in binsings, right?
21:33:05gour*bindings
21:33:09dom96I think so: http://build.nimrod-code.org/docs/libglade2.html
21:33:31gourit looks we'll use nimrod+gtk for our project...
21:33:50gourit got the best 'score' in our 'divination analyis' :-)
21:34:22Araqgour: who is this "we"?
21:34:26gourocaml was close, ada 'performed' as one could expect, while fpc/lazarus were clear NO
21:34:37Araqand how did it get the best score?
21:35:24gourAraq: because i hope some people will help me in the project...otoh, my wife and almost 10month daughter helped with 'divination'
21:35:38gourAraq: do you know about I-Ching?
21:36:01AraqI don't think I do want to know :P
21:36:11gourok then
21:37:36gourAraq: but beware, there are dangers on the nimrod-path which would have to be overcome
21:42:51AraqI'm not worried :-)
21:43:47gourgood...those should be overcome by heart, not by brain, btw. ;)
21:44:04gour...but you'll see in due course of time
21:45:46reactormonkAraq, purge ecma, replace with JS?
21:47:04Araqreactormonk: yeah
21:56:25reactormonkAraq, defined(ecmascript) => defined(JS) ?
21:56:43Araqhrm
21:57:06Araqnah
21:57:33Araqwe should introduce an alias 'JS' first
21:58:02reactormonkno deprecation for defined() stuff?
21:58:12reactormonkcan I use a term rewriting macro for that? ^^
21:58:25Araqhell no
22:33:04*gour --> sleep
22:33:09gour'night folks
22:33:13*gour quit (Quit: WeeChat 0.4.0)
22:42:02*zahary1 joined #nimrod
22:42:11*zahary1 quit (Client Quit)
23:01:45*gradha quit (Quit: bbl, have youtube videos to watch)
23:53:01*fowl quit (Ping timeout: 256 seconds)
23:54:27reactormonkAraq, so what's the idea?
23:54:41reactormonkAraq, can I somehow hook into the -d: part?