| 00:00:38 | * | Zerathul quit (Quit: ChatZilla 0.9.89 [Firefox 18.0.1/20130116073211]) |
| 01:56:42 | * | XAMPP_8 quit (Ping timeout: 252 seconds) |
| 02:04:37 | * | FreeArtMan quit (Ping timeout: 248 seconds) |
| 02:33:18 | * | fowl quit (Quit: Leaving) |
| 07:35:09 | * | zahary1 quit (Read error: No route to host) |
| 07:35:21 | * | zahary joined #nimrod |
| 08:59:07 | zahary | hi Araq |
| 09:37:34 | * | q66[bru] joined #nimrod |
| 09:55:29 | Araq | hi zahary |
| 09:56:14 | * | q66[bru] quit (Quit: Computer has gone to sleep.) |
| 09:57:46 | zahary | so, did you figure out all the details about the new GC already? |
| 09:58:33 | zahary | shall I explain where the memory increase comes from? |
| 10:01:01 | zahary | I'm getting 1.9s (old) vs 1.8s (new) for bootstrapping btw |
| 10:04:11 | Araq | isn't it obviously from the new cycle roots handling? |
| 10:04:29 | Araq | and the cycle collector is deactivated for the compiler |
| 10:04:44 | Araq | so the array needs to grow for every allocated PSym, PType |
| 10:06:07 | Araq | ambition caught me and so I'm trying hard to fix my cycle collector and not looking at your solution too much ... ;-) |
| 10:07:04 | zahary | well, I should add 2 more leaks test cases then |
| 10:08:01 | zahary | otherwise, it's not just about TCellSet vs arrays. I think the bigger contribution is something that's related to the "cycle roots trimming" that I added in one of the commits |
| 10:09:56 | Araq | I wanted to talk you about this too: |
| 10:09:59 | zahary | under the new GC, when a cycle root object is freed through the ZCT, it's not pulled out from the cycle roots immediately. instead, it's just marked dead and collected in the next CollectCycles |
| 10:10:30 | zahary | this "trimming" is the process of collecting only a small portion of the more recently dead cycle roots |
| 10:11:35 | zahary | I can add another threshold that will trigger a "full trimming" over the cycle roots and the new GC will have exactly the same behavior as the old (given that mark and sweep is disabled) |
| 10:12:08 | Araq | interesting; it seemed to try to re-invent a generational GC |
| 10:12:37 | Araq | but if you keep the the 'cycleRoots.incl' in incRef it should be possible to avoid this |
| 10:13:14 | zahary | it's a workaround for a problematic behavior manifesting when mark and sweep is disabled\ |
| 10:14:04 | Araq | the idea being that we accurately know which parts of the graph changed |
| 10:14:07 | zahary | there is no much difference if you add cycleRoots in incRef vs newObject |
| 10:14:15 | Araq | and we don't have to re-trace unchanged graphs |
| 10:14:24 | zahary | I tried both, but the numbers were only slightly off\ |
| 10:14:34 | Araq | it's a big difference |
| 10:14:54 | Araq | you can throw away all cycle roots if you have keep the incl in incRef |
| 10:15:20 | zahary | what do you mean by "throw away". I'm throwing them away after collection |
| 10:15:23 | Araq | (except the stack roots) |
| 10:16:00 | Araq | do you throw away the live roots too? |
| 10:16:17 | zahary | you mean the stack roots? |
| 10:16:55 | Araq | no stack roots, are another case |
| 10:16:58 | Araq | I have to keep those too |
| 10:17:55 | Araq | oh I see it now |
| 10:18:01 | Araq | you throw them away too |
| 10:18:05 | Araq | interesting |
| 10:18:54 | Araq | does your cycle collector work for bootstrapping? |
| 10:19:13 | zahary | what do you mean? |
| 10:19:29 | zahary | what happens if you enable it? |
| 10:19:34 | Araq | well it's disabled for the compiler itself |
| 10:19:46 | Araq | haven't tried to activate it |
| 10:19:49 | zahary | for speed, try turning it on and then claim it uses more memory :) |
| 10:21:09 | zahary | I need the new GC for the caas btw, it's a bit less cool that it's not the default now |
| 10:22:54 | Araq | hey, I thought it's fair yours isn't the default yet but has the cooler name ;-) |
| 10:23:18 | zahary | :) I can live with the v2 switch |
| 10:24:53 | zahary | Hint: operation successful (60245 lines compiled; 2.137 sec total; 139.143MB) [SuccessX] |
| 10:29:02 | Araq | so how can you throw away all cycle roots if you perform 'incl' in newObj? |
| 10:32:23 | zahary | well, why do you consider it a problem? the candidates are non-zero decrefs, new objects and previously stack referenced objects |
| 10:33:12 | Araq | new(x); fullCollect(); x.child = x; |
| 10:33:33 | zahary | x is a stack ref here? |
| 10:33:44 | Araq | no a global var |
| 10:34:11 | zahary | so, it we get RC1 treatment |
| 10:34:14 | zahary | it will get |
| 10:34:28 | Araq | ah yeah |
| 10:34:30 | Araq | hrm |
| 10:35:13 | Araq | ok, this may indeed be correct :P |
| 10:36:36 | zahary | and I have a switch for this "new objects are cycle candidates vs addCycle in incRef" |
| 10:37:56 | zahary | it's a very small factor. as I said before, the big cost of the new GC comes from the size of the whole cyclic graph |
| 10:38:45 | zahary | even 1 candidate is likely to reference the whole graph (just imagine how symbols point to their owner module, which points to its symbols, etc, etc) |
| 10:39:26 | Araq | we could make 'ower' a weak reference |
| 10:39:29 | Araq | *owner |
| 10:39:53 | Araq | (I hope) |
| 10:40:49 | zahary | yeah, we could try tweaking the types in the compiler |
| 10:43:42 | Araq | ok, next point: in the new GC you only use waPush |
| 10:44:25 | Araq | but is that really faster? |
| 10:44:43 | Araq | waZctDecRef should be better for the zct handling |
| 10:45:47 | Araq | because the temporary stack is not used then |
| 10:45:59 | Araq | for the common case of ZCT collection |
| 10:47:19 | zahary | hmm, alright, I'll try to measure this before fully moving on to waPush only markers |
| 10:47:45 | zahary | waZctDecRef has a bug in the old GC btw. you need the full write barrier there too |
| 10:48:24 | Araq | why? |
| 10:48:49 | Araq | oh hrm |
| 10:48:52 | zahary | it's the common orphaning problem. |
| 10:49:02 | Araq | yeah |
| 10:51:59 | Araq | next point: |
| 10:52:05 | Araq | for c in cycleRoots: decRefGraph(c) |
| 10:52:25 | Araq | for c in cycleRoots: if c.rc > 0: incRefGraph(c) # mark alive |
| 10:52:54 | Araq | for c in cycleRoots: if c.rc == 0: freeGraph(c) |
| 10:53:13 | Araq | is the pseudo code for the simple non-concurrent case right? |
| 10:55:39 | zahary | that's the python's article algorithm |
| 10:56:07 | zahary | it can work, but you need all cyclic object to be inside the cycleRoots (cycleRoots should be just named cyclicObjects) |
| 10:56:23 | zahary | and you should keep all survivors there |
| 10:56:26 | * | q66[bru] joined #nimrod |
| 10:57:38 | Araq | huh? I thought bacon03's algorithm ("syncronous cycle collection") is the same |
| 10:59:37 | Araq | http://www.google.de/url?sa=t&rct=j&q=bacon+et+al+reference+counting+garbage+collection&source=web&cd=1&ved=0CDAQFjAA&url=http%3A%2F%2Fresearcher.ibm.com%2Ffiles%2Fus-bacon%2FBacon03Pure.pdf&ei=XDsOUdK2L43jtQazg4GQBg&usg=AFQjCNELErgZBRbzBdOeZEvFeVyPOQL8IQ&bvm=bv.41867550,d.Yms&cad=rja |
| 11:00:28 | zahary | it's not. its contribution is the fact that only a subset of the cyclic objects can be considered candidates (and you need the proposed coloring scheme to achieve that) |
| 11:02:22 | zahary | hmm, this is not the paper I expected |
| 11:02:28 | * | FreeArtMan joined #nimrod |
| 11:02:38 | * | q66[bru] quit (Quit: Computer has gone to sleep.) |
| 11:02:51 | Araq | I'm referring to figure 7 |
| 11:02:51 | zahary | I'm talking about their "synchronous cycle collector' |
| 11:03:32 | Araq | but yeah |
| 11:03:39 | Araq | it's indeed not the same |
| 11:03:48 | Araq | which explains quite a lot ... |
| 11:03:53 | zahary | ah, yeah. fig 7 is right |
| 11:04:25 | * | FreeArtMan quit (Client Quit) |
| 11:10:05 | * | q66[bru] joined #nimrod |
| 11:16:19 | * | q66[bru]_ joined #nimrod |
| 11:17:00 | * | q66[bru] quit (Read error: Connection reset by peer) |
| 11:17:00 | * | q66[bru]_ is now known as q66[bru] |
| 11:21:12 | Araq | ok, now I know why my attempts fail ... :D |
| 11:23:00 | Araq | pity that you can't incl/excl while iterating over the TCellSet |
| 11:23:11 | Araq | keep that in mind when you play with it |
| 11:23:58 | Araq | took me a while to figure out why this has a slight chance of failing |
| 11:51:19 | * | q66[bru] quit (Quit: Computer has gone to sleep.) |
| 12:02:07 | * | FreeArtMan joined #nimrod |
| 12:02:49 | * | q66[bru] joined #nimrod |
| 12:02:51 | Araq | zahary do we really need 6 bits for all the flags? |
| 12:04:53 | * | FreeArtMan quit (Client Quit) |
| 12:49:32 | * | q66[bru] quit (Quit: Computer has gone to sleep.) |
| 12:52:51 | zahary | I'm not using one of them yet (so, they could be 5), but here is a one way of thinking about it: |
| 12:52:51 | zahary | if we worry that the counter may overflow on a 32-bit system, I calculated that just the space required for enough inbound pointers is more than 700MB and if you have such a widely shared object, then the advice "make it a global" will instantly save you that much memory :) |
| 12:54:02 | Araq | indeed that's my worry |
| 12:54:17 | Araq | 6 might be the tolerable upper bound |
| 12:55:56 | * | q66[bru] joined #nimrod |
| 12:56:11 | Araq | depending on the program it may not be easy to use a global instead |
| 12:56:41 | Araq | plus currently we don't even assert it didn't overflow |
| 12:57:01 | zahary | yes, but still 700MB for just the pointers on a machine that's capped to 3 or 4 GB |
| 12:57:09 | Araq | yeah |
| 12:57:22 | Araq | it shouldn't occur in practice |
| 12:57:45 | Araq | it's possible to construct programs that trigger it |
| 12:58:20 | zahary | I also considered the other scheme where the flags are the most significant bits |
| 12:58:32 | zahary | then overflowing just messes up with them in a safe way |
| 12:58:59 | zahary | but the though experiment was only for amusement and curiosity :) |
| 13:00:09 | Araq | lets just insert a sysAssert in incRef and it's fine with me |
| 13:00:19 | zahary | sure |
| 13:00:57 | Araq | you don't need even more bits for the concurrent collector, right? |
| 13:02:04 | Araq | so ... I guess I should live with your GC and abandon mine ... getting the cycle collector to work is more work than I expected |
| 13:02:57 | Araq | I still think we should use TCellSet for the cycle roots though |
| 13:03:23 | zahary | yes, but it may require an additional word in the TCells, although I have some ideas how to reduce the impact |
| 13:04:36 | zahary | it should be easy to try TCellSet and compare the memory use and the execution time |
| 13:05:02 | Araq | an additional word in the TCell is a show stopper |
| 13:05:10 | zahary | my previous message (about the additional word) was referring to the incremental collector |
| 13:05:14 | Araq | you may as well keep the cell seq then |
| 13:05:23 | Araq | oh I see |
| 13:08:19 | Araq | I have implemented a data structure to attach temporary RCs to a cell |
| 13:09:03 | Araq | so that no pass is necessary to restore the RCs in the cycle collection |
| 13:09:53 | Araq | but it doesn't work, because if you have a <-> b -> c <- d |
| 13:10:01 | Araq | where a and b are cycle roots |
| 13:10:29 | Araq | you can collect a, b but must keep c due to the live d |
| 13:10:52 | Araq | however, then you need to decref c |
| 13:11:02 | Araq | so you can't simply restore c's old RC |
| 13:12:04 | zahary | the additional word holds a pointer to the snapshot of the refs in the previous collection (under the sliding views algorithm). this pointer can be written in the rc fields if you also write the rc in the snapshot when it's taken. under this scheme, you pay a bit more only for the modified objects between collections |
| 13:14:07 | Araq | I dunno if the sliding view algorithm its worth its complexity |
| 13:14:43 | zahary | it's not that complex |
| 13:15:34 | zahary | we should try it (it uses the same coloring algorithm and the rest of CollectCycles will work with minor modifications) |
| 13:15:42 | Araq | well I fear debugging it ;-) |
| 13:16:04 | Araq | especially since the papers too used to have bugs |
| 13:17:48 | Araq | but sure, go ahead |
| 13:18:04 | Araq | but introduce --gc:v3 for the concurrent mode please |
| 13:21:27 | zahary | sure, after reading all these papers I got even more excited about implementing all kinds of GC within nimrod switchable at compile-time. |
| 13:22:00 | zahary | it seems that most of the GC researches are working with projects such as Jikes RVM, which is actually quite more complicated to develop for than nimrod |
| 13:22:26 | Araq | yeah we have a pretty nice foundation |
| 13:22:31 | zahary | I'd love to pitch nimrod as a GC research platfrom |
| 13:23:50 | Araq | we need more benchmark programs and tests though |
| 13:24:04 | Araq | the compiler itself is a good stress test |
| 13:24:18 | Araq | but I can't imagine making it concurrent |
| 13:24:35 | Araq | which is why with 8 cores at least, boehm's should win again |
| 13:24:41 | Araq | if it doesn't already |
| 13:25:17 | Araq | GC is pretty nice component that takes advantages of multicore even if the mutator doesn't |
| 13:26:02 | * | q66[bru] quit (Quit: Computer has gone to sleep.) |
| 13:26:39 | Araq | and the compiler also uses lots of cyclic datastructures and doesn't require responsiveness but only throughput |
| 13:29:37 | zahary | yes, there are some standard benchmarks that seem to be popular among the papers |
| 14:17:13 | * | q66[bru] joined #nimrod |
| 14:21:53 | * | q66[bru] quit (Client Quit) |
| 14:28:48 | * | q66[bru] joined #nimrod |
| 14:33:21 | Araq | dom96: the highlighter doesn't know about unsigned literals |
| 14:33:30 | Araq | 3'u8 |
| 14:33:44 | Araq | these are relatively new though |
| 14:35:46 | dom96 | ok |
| 14:51:51 | * | q66[bru] quit (Quit: Computer has gone to sleep.) |
| 14:55:50 | Araq | zahary: fyi http://www.google.de/url?sa=t&rct=j&q=bacon+et+al+reference+counting+garbage+collection&source=web&cd=21&ved=0CDAQFjAAOBQ&url=http%3A%2F%2Fwww.cis.nctu.edu.tw%2F~wuuyang%2Fpapers%2FICSOFT-31-WUUYANG-4PAGES.pdf&ei=HHoOUdupPMeitAbXyYHACA&usg=AFQjCNGmIdC_V7nP5X3PPdBxZNeU5ZlnLw&bvm=bv.41867550,d.Yms&cad=rja |
| 15:22:00 | * | XAMPP_8 joined #nimrod |
| 15:52:33 | * | Zerathul joined #nimrod |
| 15:55:11 | * | Zerathul quit (Client Quit) |
| 16:07:47 | * | gradha joined #nimrod |
| 16:25:36 | gradha | zahary: do you know how to redirect the contents of g:nim_log to a buffer/file? |
| 16:26:33 | * | q66[bru] joined #nimrod |
| 16:28:37 | gradha | I only know of redir and dumping through let, but that doesn't convert line feeds, you get everything in a single line |
| 16:32:12 | * | q66[bru] quit (Quit: Computer has gone to sleep.) |
| 18:13:44 | dom96 | wow. "compilation artifact(-14792, 3642) Error: internal error: genStmts(nkNone)" is certainly an interesting error. |
| 18:19:34 | shevy | artifacts!!! |
| 18:19:37 | shevy | dig them out, then sell them! |
| 18:20:07 | gradha | are you playing some trading game? |
| 18:20:51 | Araq | "compilation artifact" must be something zahary created :-) |
| 18:22:12 | gradha | oh, the stacktrace was so long I didn't even bother to read it all |
| 18:37:49 | * | gradha quit (Quit: bbl, have youtube videos to watch) |
| 19:15:14 | * | shevy quit (Ping timeout: 252 seconds) |
| 19:27:52 | * | shevy joined #nimrod |
| 19:48:06 | * | Boscop joined #nimrod |
| 20:12:09 | * | gradha joined #nimrod |
| 21:31:04 | Araq | gradha: is #326 really tied to macosx? |
| 21:31:37 | gradha | no idea, I don't have other platforms to test it on |
| 21:31:44 | Araq | ah ok |
| 21:48:13 | gradha | dom96: does gtk run on android? |
| 21:49:31 | gradha | yay for crazy http://code.google.com/p/android-xserver/ |
| 21:52:52 | gradha | hmm... slashdot has an article from 2011 stating gtk 3.2 will allow running apps in the browser, and the gtk guys are on 3.6.2, where's my browser gimp? |
| 21:54:51 | Araq | it helps to read fragments of your statement: |
| 21:55:02 | Araq | "slashdot has an article from 2011 stating ..." |
| 21:55:19 | Araq | :P |
| 21:58:38 | gradha | don't understand, they even had demos two years ago http://www.webupd8.org/2011/03/gtk-32-will-let-you-run-any-application.html |
| 21:58:45 | gradha | maybe they decided it was crap and ditched it? |
| 21:59:23 | dom96 | gradha: http://developer.gnome.org/gtk3/3.6/gtk-broadway.html |
| 21:59:49 | dom96 | There was a video somewhere showing it off... |
| 22:00:30 | gradha | I guess the gimp can't run just on that or requires other stuff not easily ported to the browser |
| 22:01:21 | gradha | it looks like gtk3 is the same as python3: everybody is still using 2? |
| 22:01:44 | gradha | wow, not even a single tutorial for gtk3 |
| 22:02:17 | gradha | awesome, the 2.x tutorial points to http://help.gnome.orgdevel/gtk-tutorial/stable/ |
| 22:03:01 | dom96 | Yes. And they decided to break themes in gtk3.6 or something like that. So I probably won't be porting to aporia to gtk3 anytime soon |
| 22:03:19 | dom96 | *porting aporia |
| 22:04:18 | gradha | Araq: docstrings for objects really don't work, want me to put a few cases as github issues? |
| 22:04:39 | dom96 | GTK3 apps still fail to display checkboxes properly for me. |
| 22:05:01 | dom96 | But my theme is apparently gtk3.6 compatible. |
| 22:05:23 | Araq | what do you mean, gradha ? docstrings for idetools? |
| 22:05:25 | gradha | what is this theme thing? like changing background colors or making windows with rounded corners? |
| 22:05:43 | gradha | Araq: I have an object and using docstrings for its description or the fields just doesn't cut it |
| 22:06:00 | dom96 | gradha: It's the way all widgets look. |
| 22:06:07 | Araq | type o = object # \ |
| 22:06:11 | Araq | # long text |
| 22:06:14 | Araq | # here |
| 22:06:23 | Araq | er, make that ## \ |
| 22:07:01 | gradha | that looks awesomely crap in the final html |
| 22:07:17 | Araq | how so? |
| 22:07:35 | gradha | it is copied verbatim |
| 22:07:45 | dom96 | what a positive way to say 'crap' :P |
| 22:07:57 | Araq | for the fields, yes |
| 22:08:06 | Araq | for the docs that go to the type itself, no |
| 22:08:22 | gradha | then I must be doing something wrong, I'll make a test case |
| 22:12:02 | gradha | uff... can't even imagine myself how to fix the output without rewriting it all |
| 22:13:17 | gradha | the use of verbatim text prevents anything sane due to flow issues |
| 22:13:57 | Araq | changing that is some effort |
| 22:14:14 | Araq | think of 'case' objects |
| 22:14:34 | gradha | that's precisely what I'm trying to document |
| 22:14:58 | gradha | can you have variants of variants? |
| 22:15:28 | Araq | I don't think so |
| 22:15:34 | Araq | but it's a good question |
| 22:15:49 | gradha | it would be problematic, since that would leave you with a half initialised object |
| 22:15:53 | Araq | maybe the compiler doesn't prevent that |
| 22:23:53 | NimBot | nimrod-code/Aporia 3d359a6 Dominik Picheta [+0 ±2 -0]: Fixed problems with Comment toggling messing up the undo manager. |
| 22:23:53 | NimBot | nimrod-code/Aporia 68d7526 Dominik Picheta [+0 ±3 -0]: Selecting further now clears any previous highlighted matches. |
| 22:39:44 | * | gradha quit (Quit: bbl, have youtube videos to watch) |
| 23:30:58 | * | XAMPP_8 quit (Ping timeout: 245 seconds) |