| 00:01:14 | reactormonk | client.nim(4, 28) Error: inheritance only works with non-final objects |
| 00:01:17 | reactormonk | hm |
| 00:01:24 | reactormonk | type TToplevel* = ref object <- parent |
| 00:01:34 | reactormonk | type TClient = ref object of TToplevel <- inheritance |
| 00:03:05 | Araq | the compiler is correct |
| 00:03:29 | Araq | type TToplevel* = ref object {.inheritable.} # does the trick |
| 00:03:34 | reactormonk | hm |
| 00:03:39 | Araq | or just inherit from TObject |
| 00:04:27 | Araq | btw thanks for using 'T' with 'ref'; it's against the convention :P |
| 00:04:44 | Araq | but I know I know, nobody likes it anyway |
| 00:04:45 | reactormonk | oops |
| 00:05:26 | reactormonk | sed the stuff? |
| 00:07:00 | Araq | yeah |
| 00:07:09 | reactormonk | P* instead of T*? |
| 00:15:01 | Araq | yes |
| 00:15:59 | * | q66 quit (Quit: Quit) |
| 00:37:54 | reactormonk | proc callDBus*(service, path, interface, method: string, arg: openarray[QVariant], callback: proc()) {.importc: "callDBus"} <- what's wrong here? |
| 00:37:59 | reactormonk | global.nim(14, 39) Error: ')' expected |
| 00:38:46 | Araq | good question |
| 00:38:55 | Araq | your indentation? |
| 00:45:50 | reactormonk | none |
| 00:46:13 | reactormonk | http://sprunge.us/XWdP |
| 00:47:53 | Araq | 'method' is a keyword :P |
| 00:48:09 | Araq | rename it to 'theMethod' and it should work |
| 00:52:09 | reactormonk | nope |
| 00:55:29 | reactormonk | Araq, interface is a keyword too :-/ |
| 00:55:36 | Araq | ha ah yeah |
| 00:55:52 | Araq | I thought it's a good idea to reserve it for future growth :P |
| 00:56:19 | reactormonk | I agree |
| 00:56:37 | Araq | yeah of course you do |
| 00:57:00 | reactormonk | should I disagree? :-P |
| 00:57:14 | Araq | in fact yes |
| 00:57:25 | Araq | that would be refreshing |
| 00:57:41 | Araq | it seems I'm the only one left who hates interfaces ... |
| 01:04:28 | reactormonk | how else do you deal with iterators? |
| 01:05:05 | NimBot | nimrod-code/Aporia ac1262c Dominik Picheta [+0 ±2 -0]: Languages in the View -> Syntax Highlighting menu are now sorted... 2 more lines |
| 01:05:05 | NimBot | nimrod-code/Aporia 8fc006f Dominik Picheta [+0 ±3 -0]: Improved search, and fixed a slight bug with backwards search. |
| 01:05:28 | dom96 | good night |
| 01:06:56 | * | XAMPP-8 quit (Read error: Connection reset by peer) |
| 01:09:33 | NimBot | Araq/Nimrod c299eb1 Araq [+0 ±5 -0]: small bugfix for lambdalifting; preparations for a better testing framework |
| 01:22:56 | reactormonk | https://gitorious.org/learn-x-the-hard-way/ ? |
| 01:23:50 | Araq | good night |
| 02:13:41 | * | XAMPP-8 joined #nimrod |
| 02:23:19 | * | fowl quit (Quit: Leaving) |
| 04:33:44 | * | zahary quit (Ping timeout: 260 seconds) |
| 04:55:02 | * | zahary joined #nimrod |
| 05:10:59 | * | XAMPP joined #nimrod |
| 05:10:59 | * | XAMPP quit (Changing host) |
| 05:10:59 | * | XAMPP joined #nimrod |
| 06:03:18 | * | ccssnet quit (Quit: http://atccss.net) |
| 06:20:31 | * | ccssnet joined #nimrod |
| 06:58:05 | * | zahary quit (Ping timeout: 252 seconds) |
| 07:02:59 | * | zahary joined #nimrod |
| 08:11:23 | * | XAMPP-8 quit (Ping timeout: 245 seconds) |
| 09:33:29 | * | zahary quit (Ping timeout: 240 seconds) |
| 09:35:35 | * | zahary joined #nimrod |
| 10:09:18 | * | fowl joined #nimrod |
| 11:07:58 | * | codeallergy joined #nimrod |
| 11:09:01 | dom96 | good morning |
| 11:09:10 | Araq | hi dom96 |
| 11:09:17 | Araq | I didn't break the tester :-) |
| 11:09:54 | dom96 | great :D |
| 11:13:10 | * | q66 joined #nimrod |
| 12:04:43 | Araq | dom96: please push your 'do' bugfix ;-) |
| 12:04:48 | Araq | I want to test it |
| 12:05:04 | dom96 | I haven't tested it well enough though. |
| 12:05:24 | Araq | well do that then |
| 12:06:05 | dom96 | After I finish this feature i'm working on in Aporia I will |
| 12:06:59 | Araq | alright thanks |
| 12:37:06 | * | FreeArtMan joined #nimrod |
| 13:24:42 | * | Camille_ joined #nimrod |
| 13:25:03 | Camille_ | ok |
| 13:25:21 | dom96 | hello Camille_ |
| 13:25:22 | Camille_ | I did it |
| 13:25:33 | Camille_ | hello dom |
| 13:25:51 | dom96 | You did what? |
| 13:26:10 | Camille_ | dang Im in the wrong room lmao |
| 13:26:19 | Camille_ | well I was looking fro nimrod |
| 13:26:29 | Camille_ | I downloaded hex chat |
| 13:26:45 | dom96 | LIke, the person? :P |
| 13:26:52 | Camille_ | yes |
| 13:27:00 | dom96 | /msg Nimrod hello |
| 13:27:20 | Camille_ | well Im on mIRC so Ill do that from there hmm |
| 13:27:32 | Camille_ | weird |
| 13:27:38 | Camille_ | I was in his room |
| 13:27:47 | Camille_ | and its the same #nimrod |
| 13:27:51 | * | Camille_ shrugs |
| 13:28:01 | Camille_ | anyway |
| 13:28:04 | Camille_ | thx |
| 13:28:07 | Camille_ | and sorry |
| 13:28:18 | dom96 | no worries |
| 13:30:09 | * | Camille_ left #nimrod ("Leaving") |
| 14:38:31 | * | FluffySauce joined #nimrod |
| 14:39:15 | Araq | hi FluffySauce |
| 14:39:29 | FluffySauce | Hello |
| 14:40:58 | dom96 | FluffySauce: welcome to #nimrod! |
| 14:41:32 | FluffySauce | Thanks! I'm just now exploring the language :) It's quite nice |
| 14:43:09 | dom96 | Nice to hear :) |
| 14:50:01 | * | FreeArtMan quit (Quit: Leaving) |
| 16:15:28 | * | FluffySauce quit (Quit: Textual IRC Client: www.textualapp.com) |
| 16:34:28 | Araq | ping reactormonk |
| 18:27:14 | * | shevy quit (Ping timeout: 252 seconds) |
| 18:30:35 | reactormonk | pongy Araq |
| 18:31:58 | Araq | reactormonk: never mind ;-) |
| 18:32:11 | Araq | wanted you to ask if you have any exp with greasemonkey |
| 18:32:26 | Araq | but I figured I don't want to use it ;-) |
| 18:33:25 | reactormonk | Araq, for the record: nope |
| 18:35:48 | Araq | alright |
| 18:36:29 | Araq | ping zahary |
| 18:37:36 | Araq | dom96: how's that regex leak going? |
| 18:38:39 | dom96 | well in the past 2 hours I have watched Fringe, and eaten some nice chinese food. So it's not going at all :P |
| 18:39:37 | * | shevy joined #nimrod |
| 18:43:09 | zahary | hi Araq |
| 18:43:46 | Araq | so ... is MIT license fine with you, zahary? |
| 18:44:17 | zahary | yes, I don't mind the change |
| 18:45:02 | Araq | great |
| 18:50:28 | Araq | so zahary when will your awesome GC fixes arrive? ;-) |
| 18:51:25 | reactormonk | zahary, and your idetools daemon? |
| 18:56:07 | zahary | I really hoped to do some work today, but I'm in my usual procrastination routine - reading stuff and chasing links all day |
| 18:56:39 | Araq | hah, you know what helps? |
| 18:56:59 | Araq | plugging the LAN :P |
| 18:57:46 | zahary | heheh, my ISP seems to care about my productivity, because the internet connection goes away from time to time :) |
| 18:59:13 | dom96 | good guy ISP :D |
| 19:01:12 | Araq | zahary it's off topic but ARM's new multi core architecture uses a memory model that's compatible with C++11's |
| 19:01:37 | Araq | which means somebody should read all the details about it :P |
| 19:02:35 | Araq | and IMO it also means they got it right |
| 19:03:42 | zahary | http://herbsutter.com/2012/08/02/strong-and-weak-hardware-memory-models/ is the difference about this "weak vs strong" model or there is more? |
| 19:04:40 | Araq | nah I think that captures the essence of it |
| 19:12:32 | zahary | speaking of new CPUs, the next generation of Intel's CPUs will have support for hardware transaction memory, which should create bigger waves in the software side of things |
| 19:15:30 | Araq | link? that news escaped me |
| 19:15:44 | Araq | I thought transactional memory is kind of dead now ... |
| 19:16:40 | zahary | http://arstechnica.com/business/2012/02/transactional-memory-going-mainstream-with-intel-haswell/ |
| 19:17:31 | zahary | http://www.realworldtech.com/haswell-tm/3/ |
| 19:18:11 | Araq | I'm scared of STM ;-) |
| 19:18:35 | Araq | I fear it'll replace deadlocks with livelocks in practice ... |
| 19:18:46 | Araq | good luck finding *those* |
| 19:18:58 | Araq | whereas deadlocks are pretty easy to debug |
| 19:24:00 | zahary | I'm not sure whether it will be widely teached/used or it will become something similar to the lock-free algorithms. i.e. some data structures written by experts use it for added performance, but ordinary people are told to stay away from it |
| 19:24:30 | zahary | it really depends how the hardware will perform actually (how much memory can be touched in transaction, what is the cost, etc) |
| 19:24:50 | Araq | yeah but Intel's implementation of the concept looks sweet |
| 19:25:11 | Araq | it's basically only a form of optimistic locking |
| 19:25:19 | Araq | to the software it's just like a lock |
| 19:25:29 | Araq | unless you enter the RTM mode |
| 19:27:15 | zahary | yeah, I've forgotten some of the details, but I remember the gist of what you mean |
| 19:28:11 | zahary | take a look at the paper for the proposed C++ integration btw - we can copy the constructs from there |
| 19:30:08 | Araq | iirc 'atomic' is still a keyword in nimrod ;-) |
| 19:33:33 | Araq | from a language design point of view 'lock' and 'atomic' are very similar; the hard part is to ensure statically any access to 'shared' data is enclosed in a lock/atomic section |
| 19:33:53 | Araq | aka race freedom |
| 19:36:54 | Araq | in fact there is paper by Hans Boehm arguing that atomic's semantics should be 'use a single global lock' and the implementation should optimize it to use transactions instead |
| 19:37:19 | Araq | this way 'amotic' is well defined wrt IO |
| 19:37:22 | Araq | *atomic |
| 19:38:34 | zahary | well, there are practical differences too - performance, used resources, etc - so the user should be able to specify both an abstract requirement like "atomic" and specific requirement like "atomicity using a inter-process named mutex" |
| 19:40:55 | Araq | that's stuff for pragmas affecting 'atomic' |
| 19:41:31 | Araq | we can implement 'atomic' as a single global lock and claim it's QOI issue :P |
| 19:43:10 | Araq | as I said the next steps are to implement 'shared' and make the compiler moan if you access shared data not in an atomic section |
| 19:45:27 | Araq | fyi Go only runs single threaded in google's app store |
| 19:45:46 | zahary | one difference between transactions and lock-guarded atomic blocks btw is that transactions also add a rollback operation |
| 19:45:52 | Araq | because they can't guarantee memory safety otherwise :D |
| 19:46:35 | Araq | a race in Go can break everything afaik |
| 19:46:46 | Araq | it's much worse than Java in this respect |
| 19:46:59 | zahary | why so? is this coming from the GC? |
| 19:47:29 | Araq | no I think they're quite permissive in their memory modell when it comes to races |
| 19:47:55 | Araq | for performance reasons |
| 19:48:14 | Araq | Java is stricter and so can keep safety |
| 19:48:49 | zahary | but you can still use regular mutexes, etc to get safety? |
| 19:49:35 | Araq | sure but the app store is about running untrusted code |
| 19:50:21 | Araq | dunno why they don't use ordinary process sandboxing for it |
| 19:50:29 | Araq | maybe they do |
| 19:50:49 | zahary | you are talking about the web server that implements the store? |
| 19:51:12 | zahary | aren't you talking ... |
| 19:52:12 | Araq | http://stackoverflow.com/questions/3800435/why-does-google-app-engine-support-a-single-thread-of-execution-only |
| 19:52:27 | zahary | ah, the app engine |
| 19:56:19 | Araq | and I know about the rollback operation of a transaction; I'm not too exicted about this though |
| 19:57:18 | Araq | you can't rollback an 'echo' statement so the most common action is to retry |
| 19:58:15 | Araq | and then the semantics are lock-like |
| 19:58:25 | zahary | strictly speaking, echo within a transaction will be probably forbidden under the proposed C++ rules |
| 19:58:55 | Araq | true in my mind 'atomic' only allows pure memory operations |
| 19:59:04 | zahary | they add some effects like "transaction-safe" that are computed for functions and enforced in atomic blocks |
| 19:59:56 | Araq | our effect system could even allows for FReadIO in a transaction :-) |
| 20:00:58 | zahary | yep, but we should be careful to make the underlying C compiler happy too |
| 20:03:24 | Araq | you can always cast the safety away in C++ |
| 20:04:08 | Araq | maybe better to play nice with C++'s effect system though |
| 20:07:06 | zahary | the proposal is not accepted yet and I'm sure there will be a C interface as well so we should just wait and see |
| 20:07:51 | Araq | yeah well our next steps are not affected by it much |
| 20:08:14 | Araq | we need 'shared' as a type modifier and a shared GC |
| 20:08:43 | zahary | did you read the heap sliding views paper btw? |
| 20:08:53 | Araq | and perhaps a lockfree memory allocator |
| 20:09:00 | Araq | no sorry, didn't read them |
| 20:09:11 | Araq | read about lockfree memory allocators instead |
| 20:09:26 | Araq | though I'm not sure how they work tbh :-) |
| 20:10:58 | zahary | haven't been there - I studied lock-free reference counting and was quite turned off by the complexities/costs |
| 20:11:23 | Araq | that's not suprising |
| 20:11:46 | zahary | I was about to copy the boost::shared_ptr's spinlock approach as a result |
| 20:11:55 | Araq | why? |
| 20:12:10 | Araq | just use 'lock sub' and call it a day |
| 20:12:39 | zahary | well, because basically lock-free reference counting looks very similar to some nested use of spinlocks |
| 20:12:59 | Araq | spinlocks and lockfree are different |
| 20:13:00 | zahary | lock sub? what do you mean |
| 20:13:09 | Araq | took me a while to figure out the difference |
| 20:13:19 | Araq | lets see if I can explain it |
| 20:13:28 | zahary | I remember a link you posted here |
| 20:13:39 | Araq | (lock sub -- x86's asm; 'atomicDec' in Nimrod) |
| 20:13:56 | zahary | ah, this doesn't work unfortunately |
| 20:14:08 | Araq | why not? |
| 20:14:37 | Araq | you need some proper locks later for accessing the cycle root structure etc. |
| 20:14:54 | zahary | there is a possible thread preemptions that results in incorrect counts |
| 20:14:55 | Araq | but apart from that I can't see any problem with the approach |
| 20:15:18 | Araq | Delphi uses it everywhere |
| 20:16:04 | zahary | it safe to do atomicInc when cloning a ptr and atomicDec when nullifying one |
| 20:16:12 | zahary | but it's not safe in pointer assignment |
| 20:16:34 | Araq | it is safe I think |
| 20:17:04 | Araq | especially since it's added to the ZCT which removes entries with RC > 0 later |
| 20:17:25 | Araq | so even if it's temporarily 0 it shouldn't matter |
| 20:21:38 | Araq | multithreaded reference counting is used everywhere; lots of C++ libraries use it too |
| 20:22:00 | Araq | can't see why it should require lockfree programming |
| 20:22:39 | zahary | that's not the problem. the problem is the pointer exchange should be in the same atomic operation as the reference count updates |
| 20:22:50 | * | XAMPP-8 joined #nimrod |
| 20:23:40 | Araq | I don't think that's necessary |
| 20:24:07 | Araq | atomicInc source |
| 20:24:13 | Araq | atomicDec dest |
| 20:24:14 | zahary | let A, B and C be 3 different pointer values and you have concurrent updates to 2 pointeres |
| 20:24:14 | zahary | A = B (first pointer); A = C (second poiner) |
| 20:25:00 | Araq | ok |
| 20:25:02 | zahary | the first one gives you dec A; inc B; and the second one gives you dec A; inc C; |
| 20:25:18 | zahary | I meant 2 threads accessing the same pointer actually (not 2 different pointers) |
| 20:25:42 | Araq | no, it gives: (inc B; dec A) || (inc C; dec A) |
| 20:26:24 | zahary | ok, now try to imagine that the B assignment will trump over the C assignment |
| 20:26:45 | zahary | we can end up doing inc C, but without any pointer to C remaining |
| 20:28:11 | Araq | inc B; dec A; A = B || inc C; dec A; A = C --> |
| 20:28:53 | Araq | give a linearization that breaks it please |
| 20:29:46 | zahary | inc B; dec A; inc C; dec A; A = B; A = C |
| 20:30:02 | zahary | http://hacksoflife.blogspot.com/2009/07/cas-and-reference-counting-dont-mix.html |
| 20:35:44 | Araq | so ... what's the problem? |
| 20:36:08 | zahary | "multithreaded reference counting is used everywhere; lots of C++ libraries use it too" - see my previous comment about boost::shared_ptr - this is the most famous C++ library and I was about to copy its approach |
| 20:37:20 | zahary | well, don't you see the problem now? we ended up doing 2 decrefs on A instead of one, and we incref-ed C when we should have left it unchaned |
| 20:38:04 | Araq | I see multiple problems so I want to ensure I got it ;-) |
| 20:43:29 | Araq | so I guess that means Delphi's strings are not entirely thread safe ... |
| 20:43:57 | Araq | as they don't use a lock around the string assignments iirc |
| 20:45:48 | Araq | however I wonder if it's not possible somehow |
| 20:48:16 | zahary | do you have the sources? some code archeology may help |
| 20:49:50 | Araq | I'm looking for it right now |
| 20:53:49 | Araq | x := y; where x and y are strings produces: |
| 20:53:59 | Araq | movq U_P$STRINGTEST_Y,%rdi |
| 20:54:01 | Araq | call FPC_ANSISTR_INCR_REF |
| 20:54:02 | Araq | movq $U_P$STRINGTEST_X,%rdi |
| 20:54:04 | Araq | call FPC_ANSISTR_DECR_REF |
| 20:54:06 | Araq | movq U_P$STRINGTEST_Y,%rax |
| 20:54:07 | Araq | movq %rax,U_P$STRINGTEST_X |
| 20:54:56 | Araq | which is (inc; dec; asgn) |
| 20:57:54 | zahary | are these atomic inc/dec at all? |
| 20:58:17 | Araq | I think so I'm looking for them again |
| 21:00:55 | Araq | yeah the source uses declocked/inclocked |
| 21:01:58 | Araq | I guess they declare concurrent writes to the same string as unsupported :P |
| 21:03:56 | Araq | and we can do the same: if any write access to a shared type has to be within a lock/atomic statement anyway the write barrier does not need any further locking |
| 21:06:06 | zahary | well, yeah, that's an option, but you should also check the sliding view algorithm - it prevents the whole problem by not requiring refcount updates in pointer assignments |
| 21:07:17 | Araq | what about: inc B; tmp = A; A = B; dec tmp ? |
| 21:07:52 | Araq | or even: tmp1 = B; tmp2 = A; A = B; inc tmp1; dec tmp2 ? |
| 21:08:04 | Araq | are you sure these break too? |
| 21:08:31 | zahary | I think so, let me try to find the bad sequence |
| 21:09:36 | Araq | not using refcount updates in pointer assignments is unsexy; I tried that long ago |
| 21:09:50 | Araq | it was so expensive it's not even funny |
| 21:10:51 | zahary | you tried the algorithm from the first paper, not the fancier algorithm from the new papers |
| 21:11:13 | Araq | alright let me have a look at the paper |
| 21:23:25 | Araq | http://www.google.de/url?sa=t&rct=j&q=reference%20counting%20gc%20sliding&source=web&cd=4&sqi=2&ved=0CFMQFjAD&url=http%3A%2F%2Fciteseerx.ist.psu.edu%2Fviewdoc%2Fdownload%3Fdoi%3D10.1.1.155.490%26rep%3Drep1%26type%3Dpdf&ei=ugz7UJOLFMfJswb4zoDQBQ&usg=AFQjCNFrGdgBTHgqTbFvh4KCaXIzbRKQjw&bvm=bv.41248874,d.Yms&cad=rja |
| 21:23:32 | Araq | that's it, right? |
| 21:25:44 | Araq | argh, wrong authors |
| 21:27:45 | zahary | http://www.research.ibm.com/people/d/dfb/papers/Paz03OnTheFly.pdf |
| 21:28:29 | zahary | there are also some later papers with further refinements, but the basic idea is presented here |
| 21:35:14 | Araq | there is no pseudo code :-/ |
| 21:35:43 | Araq | however they refer to Levanoni's work |
| 21:35:56 | Araq | which is my link above |
| 21:38:12 | Araq | however they still do: buffer[currPos++] = (slot, rc) |
| 21:38:19 | Araq | where buffer is thread local |
| 21:38:33 | Araq | they have a dirty flag to prevent multiple entries in the buffer |
| 21:39:02 | Araq | but that's quite similar to what I implemented and its performance was pretty poor |
| 21:40:30 | Araq | perhaps it's worth a try though |
| 21:41:20 | Araq | btw what these papers often don't talk about: |
| 21:41:36 | * | gradha joined #nimrod |
| 21:41:43 | Araq | what happens once the buffer is full? a collection needs to be done then |
| 21:41:58 | Araq | this means you have a collection every N pointer modifications |
| 21:42:15 | Araq | in this case with the dirty flag, after N modifications to different pointers |
| 21:43:46 | zahary | they do talk about this. at most double the memory is used if you snapshot every single object in the world |
| 21:44:04 | Araq | I'm not talking about memory usage |
| 21:44:11 | Araq | I'm talking about number of collections |
| 21:44:28 | Araq | conservative stack scanning is not free |
| 21:44:29 | zahary | you don't need to have a fixed size shapshot buffer, you can allocate these dynamically |
| 21:44:49 | Araq | that's still worrying me |
| 21:45:33 | Araq | inplace RC updates don't have that problem |
| 21:46:31 | zahary | what seems nice to me is that after some time, most modified objects are in the snapshot and the barrier potentially becomes very cheap (just a single if) |
| 21:46:55 | zahary | but there is one big problem with the algorithm. you haven't cited it yet :) |
| 21:48:34 | Araq | stop the world for collection? |
| 21:48:50 | Araq | don't make me guess; just tell me :P |
| 21:49:15 | Araq | hi gradha, I assume you're fine with a license change to the MIT license? |
| 21:50:44 | zahary | the first thing to note is that you're not putting the modified pointer in the snapshot, you are putting its enclosing object - this works fine for most objects, but it will be quite inefficient for sequences |
| 21:51:13 | gradha | yes |
| 21:53:00 | zahary | I spent some time thinking wether you still can use inline reference counting for refs within a sequence, but unfortunately this will break the incremental path in the algorithm |
| 21:55:46 | Araq | ah yeah another good point |
| 21:56:51 | zahary | I also considered breaking the sequence in smaller chunks that will be put in the snapshot (similar to card marking), but this is somewhat problematic too for inserts within the sequence. finally, as an extreme in this direction it's possible to just store a dirty bit within the ref or next to the ref, but these are not that great either |
| 21:58:04 | Araq | er ... wait a second |
| 21:58:32 | Araq | why is that a problem? we access the RC directly not caring about interior pointers already |
| 21:58:50 | Araq | we may as well access a 'dirty' flag instead |
| 21:59:35 | zahary | I don't follow |
| 22:00:12 | Araq | " you're not putting the modified pointer in the snapshot, you are putting its enclosing object" -- what do you mean exactly? |
| 22:00:51 | Araq | the pointer points to the start of the object |
| 22:00:59 | Araq | so that should be a problem |
| 22:01:08 | zahary | if you have a type like type Foo = object x: ref Bar; y: ref Baz |
| 22:01:37 | Araq | *shouldn't be a problem |
| 22:01:51 | zahary | foo.x = someBar; this checks the dirty flag of foo, and writes both the x and y fields in the snapshot |
| 22:02:41 | zahary | so far so good? |
| 22:03:19 | Araq | I see |
| 22:04:31 | zahary | so it should be obvious now why seq[ref Bar] is a problematic case |
| 22:05:57 | Araq | dunno for card marking you can either store the pointer to the field or the pointer to its surrounding object |
| 22:06:10 | Araq | so I assume you can do the same for this algorithm |
| 22:07:08 | Araq | that obj.field = x can be split up due to 'var' in Nimrod is a problem for many GC papers out there |
| 22:07:58 | Araq | or a design problem of Nimrod depending on how you look at the problem |
| 22:08:13 | zahary | finding the "root" of the allocation is another aspect of this, but I've already worked out the details about it (and implemented some of it) |
| 22:09:00 | zahary | clarify what you meant by "so I assume you can do the same for this algorithm" |
| 22:09:39 | zahary | the goal of the snapshot is to have an accurate view of what the heap looked like in the last collection - such a view is required for the algorithm to work |
| 22:10:26 | Araq | I can't clarify; it's just that it looks very similar to a write barrier for any other collector |
| 22:11:32 | Araq | I also can't follow; finding the root of an allocation is trivial; it's already implemented for stack scanning |
| 22:11:57 | zahary | well, most of the time it can be statically determined |
| 22:12:04 | zahary | that's what I've implemented |
| 22:12:24 | Araq | I don't like that in the write barrier though |
| 22:12:50 | zahary | you don't like what? |
| 22:13:10 | Araq | to account for interior pointers in the write barrier |
| 22:13:32 | Araq | the split var/ptr/ref is all about avoiding it |
| 22:13:36 | zahary | as I said, I statically determine the root in most of the usual situations |
| 22:13:51 | zahary | with var arguments it's required to have dynamic lookup |
| 22:16:21 | Araq | does that mean you support safeAddr(x): ref T ? |
| 22:16:34 | gradha | it seems that the new mega site (http://arstechnica.com/business/2013/01/mega-arrives-ars-goes-hands-on-with-kim-dotcoms-cloud-storage-site/) will nicely complement github's lack of public downloads |
| 22:16:48 | gradha | 50GB, that's plenty of nimrod stable versions |
| 22:17:33 | Araq | gradha: in fact we have a server already for nimrod-code.org and the downloads there |
| 22:17:54 | Araq | it's not like we need github's download feature |
| 22:17:57 | zahary | your question is whether such a public functions should exist? I don't think so |
| 22:18:21 | Araq | ok then |
| 22:19:05 | Araq | I'd rather see nil check elimination for the write barrier though |
| 22:19:29 | Araq | the new not-nil stuff with construction expressions supports that easily |
| 22:19:45 | zahary | I haven't implemented everything about these write barriers yet - only the part I considered harder: having an expression like foo.bar[213].baz = x, accurately figuring out which heap object is being written to |
| 22:21:02 | zahary | but we also changed the discussion topic, the problem we should be talking about is the seq[ref Bar] case |
| 22:22:08 | Araq | why exactly? do you still want to implement this GC? |
| 22:22:36 | Araq | I'd go with a simple stop the world GC that requires pointer updates in lock/atomic sections |
| 22:23:48 | zahary | having read everything about it and the java guys results, I think it has the potential to be faster than the atomic refcounts |
| 22:24:15 | Araq | but it's not about being fast in version 1 |
| 22:24:16 | zahary | it has other advantages too - you'll add smaller number of cycle candidates, etc |
| 22:24:32 | Araq | currently we have *no* shared heap GC |
| 22:25:35 | Araq | and btw the current implementation does stuff like 'newObjRC1' to avoid the write to the ZCT array |
| 22:25:44 | Araq | and it payed off |
| 22:26:08 | Araq | so everything that requires adding to an external array looks suspicious for performance for me |
| 22:28:48 | zahary | well, why are we considering writing to an array as some mysterious operation with unclear cost - it involves a potential allocation, an increment, and a store through a pointer. |
| 22:30:08 | Araq | maybe you can try to measure how many different objects are updated between 2 GC cycles |
| 22:30:27 | Araq | to get some number of the cost |
| 22:33:07 | zahary | sure, we should test it - I'm not making another point clearly - the sliding view algorithm is pretty much the same as the current one sans the different write barriers and a small difference how "for each field" is implemented in the CollectCycles body |
| 22:33:47 | zahary | so it's not very hard to get some experiments/tests about it |
| 22:35:21 | Araq | the real question is: now that we have a good realtime GC for thread local collections (we do, right?), what do we need for shared heap collections? |
| 22:35:44 | Araq | in the context of gaming ;-) |
| 22:36:30 | Araq | and that's mostly up to your experience to decide |
| 22:37:38 | zahary | unfortunately, the small pauses are gone now - CollectCycles in the compiler can take a lot of time |
| 22:37:59 | Araq | that doesn't worry me at all |
| 22:38:04 | zahary | another benefit of the sliding view algorithm is that's incremental |
| 22:38:34 | Araq | even the old CollectCycles was always missing the deadlines |
| 22:38:54 | Araq | deactivate it and call it when appropriate for realtime |
| 22:39:00 | Araq | applications |
| 22:39:10 | zahary | well, what's the appropriate time in a real-time game? |
| 22:39:25 | Araq | quick save :P |
| 22:40:28 | zahary | we are really stretching the definition of "real-time" here |
| 22:40:52 | Araq | that's why we call it "soft real-time" :P |
| 22:42:06 | zahary | and you blame D of marketing talk :) we should try to get some incremental algorithm running |
| 22:42:21 | gradha | can you run the collector while an IO operation is being performed and the user process is waiting? |
| 22:44:06 | zahary | the users could do that themselves, but this is still a pause and thus not appropriate in some situations |
| 22:44:44 | gradha | yes, streaming audio in the background from disk would come as a side question |
| 22:45:41 | zahary | there are usually async APIs for audio that you use from your main game thread |
| 22:48:05 | Araq | I never claimed more than what gc.txt says about soft realtime operation |
| 22:49:01 | Araq | and hard realtime is a completely different topic |
| 22:49:36 | Araq | you have things like virtual memory and caches and an OS not caring about your realtime desires |
| 22:50:04 | Araq | "hard realtime" is not what games are about |
| 22:50:14 | * | codeallergy quit (Read error: Connection reset by peer) |
| 22:50:38 | Araq | hard realtime is very expensive to achieve and doesn't mean "efficient" either |
| 22:50:47 | Araq | it means to provably not miss your deadlines |
| 22:51:14 | zahary | yes, but soft realtime will imply "no long pauses" in my book. no compromises like quick saves, level loading screens, etc |
| 22:51:39 | Araq | but it all depends on how often you need to run the cycle collector |
| 22:51:54 | Araq | running it and missing some frames is acceptable if it doesn't run too often |
| 22:54:35 | zahary | I don't know what are we arguing about. I'm sure you are not suggesting that we should never implement an incremental GC as a compile-time option and I'm not suggesting that we need to do this immediately. I'm just saying that it's not that out-of-reach and we should strive for it |
| 22:55:06 | Araq | well I'm defending my marketing claims ;-) |
| 22:56:31 | Araq | and of course I don't mind a fully incremental GC |
| 22:57:06 | zahary | alright, then. most game developers will avoid cyclic structures in the first place I think :) |
| 22:57:21 | Araq | that's more or less my point |
| 22:57:46 | fowl | i got good results with nimrods gc |
| 22:57:50 | gradha | most game developers will also pseudo-manage their memory statically by preallocating huge arrays of enemies, bullets, whatever |
| 22:58:03 | dom96 | zahary: Just remembered: gradha encountered a bug with gcc on Mac OS X, at the time Araq said that it might have been fixed by you. Have you fixed any bugs to do with gcc on Mac OS X? |
| 22:58:17 | fowl | even using refs for particles in a particle sys and spamming them didnt create big slowdowns |
| 22:58:21 | zahary | nope, I'm using clang myself |
| 22:58:54 | zahary | fowl, do you have any cyclic structures? |
| 22:59:41 | Araq | I thought the bug is about the GC freeing some growing sequence prematurely |
| 22:59:53 | gradha | dom96: if zahary doesn't have a fix for gcc maybe we should force nimrod to use clang anyway, since all users will wonder what happens |
| 23:00:05 | gradha | at least for macosx |
| 23:00:07 | fowl | not if i can help it |
| 23:00:27 | zahary | how do I trigger the bug? |
| 23:00:32 | fowl | zahary: https://github.com/fowlmouth/nimrod-sfml/blob/master/examples/particles.nim |
| 23:01:22 | dom96 | gradha: Sure, that sounds like a good solution. What do you think Araq? |
| 23:01:34 | gradha | zahary: for me at least compiling nimrod with gcc when I try to run any program more complex than hello world it stack traces randomly at different places |
| 23:02:02 | Araq | gradha: that indeed sounds like some non-GC related issue |
| 23:02:25 | gradha | I've never looked too much into it since using clang "fixes" it |
| 23:03:22 | gradha | also, when using gcc kock entered an infinite loop at some point |
| 23:03:22 | gradha | that's why I couldn't use the mac for the build farm |
| 23:03:24 | gradha | again, koch worked a few times, then stopped |
| 23:03:45 | gradha | I guess that's what they call heisenbug |
| 23:04:10 | zahary | alright, I'll take a look. I remember switching to clang after 10.8 came out, because I had some issues with gcc back then. It was using gcc for a long time under 10.6 |
| 23:05:23 | gradha | oh, I wrote kock <giggles> |
| 23:06:24 | gradha | let's try again the build farm! what was the irc channel name? |
| 23:07:18 | Araq | dom96: I don't think switching to clang is an option when it's instead some lurking codegen/GC bug in nimrod |
| 23:09:23 | gradha | here it goes, stuck in "koch clean", already got 1 minute of cpu time for itself |
| 23:12:56 | gradha | bah, killed, no point in making heat |
| 23:19:06 | * | gradha quit (Quit: Leaving) |
| 23:20:05 | Araq | ugh ... I was about to ask him for a stack trace ... |
| 23:21:12 | fowl | jesus mac is on 10.8 now? |
| 23:23:54 | Araq | oh btw fowl I'm working on fixing that bug that keeps keineschweine from compiling |
| 23:24:06 | Araq | but you need to update your code |
| 23:24:31 | fowl | ok |
| 23:24:33 | Araq | {.pop: cdecl.} was never valid for instance |
| 23:27:05 | fowl | hrm i need to figure out how to update the submodules |
| 23:27:20 | Araq | never use git submodules |
| 23:47:10 | Araq | zahary: thinking about it the A= B || A = C example is a classical write write conflict |
| 23:47:32 | Araq | so arguably it doesn't need to be supported |
| 23:47:53 | Araq | are there other cases where the atomic inc/dec solution breaks? |
| 23:53:19 | fowl | Araq: i get Error: invalid pragma: cdecl with {.pop cdecl.} |
| 23:53:50 | Araq | fowl: {.pop.} is enough |
| 23:54:04 | Araq | the compiler knows what to pop off its internal stack |
| 23:57:05 | fowl | Araq: i have a lot to clean up in here ill probably have it done by tomorrow tho.. maybe |
| 23:58:03 | Araq | the version I have requires 2 small fixes to trigger the internal compiler error |
| 23:58:35 | Araq | the other change is to use {.push callConv:cdecl.} instead of {.push: cdecl.} |
| 23:58:35 | fowl | mostly i need to change references to sfml* to csfml* |
| 23:58:48 | fowl | oo ok |
| 23:58:55 | fowl | i thought {.push cdecl.} was enough |