<< 09-11-2014 >>

00:01:11*Trixar_za quit (Ping timeout: 265 seconds)
00:04:16*Matthias247 quit (Read error: Connection reset by peer)
00:04:23*uber joined #nimrod
00:04:23*uber quit (Max SendQ exceeded)
00:04:25*Trixar_za joined #nimrod
00:04:53*uber joined #nimrod
00:17:44*uku quit (Ping timeout: 255 seconds)
00:23:17Varriountdom96: ping
00:23:55dom96Varriount: Please just ask me instead of pinging.
00:24:46VarriountOk. dom96: Does the VPN have any firewall or file which needs to be altered to let traffic through to a particular port?
00:25:30VarriountI have the builder listening on all interfaces, at port 8010, yet when I connect to nim-lang.org:8010, I get a 'ERR_CONNECTION_REFUSED' status in my web browser.
00:25:54dom96nim-lang.org does not point to that IP
00:26:20*kniteli joined #nimrod
00:27:11dom96No, you do not need to do anything for that port to work.
00:29:21AraqVarriount: any luck with the "nim picks the wrong stdlib" issue?
00:31:24VarriountAraq: I've been working on the buildbot, so no.
00:31:28*BitPuffin quit (Ping timeout: 244 seconds)
00:33:56VarriountAraq, dom96: http://178.62.143.63:8010/waterfall
00:34:17Jehan_Araq: Where does the issue show up, is it specific to some OS or a general thing?
00:35:21AraqJehan_: I had it on windows when testing something else
00:35:33Jehan_Hmm.
00:35:42Araqand from lots of reports here in #nimrod I suspect it's a general issue
00:35:54VarriountIt's never effected me.
00:35:57Jehan_For what it's worth, when I ran ./koch test, it picked the nim from my PATH rather than the one in bin
00:35:59Varriount*affected
00:36:16Jehan_But I think that's something different.
00:36:32Araqwell I dunno. I'm installing 0.9.2 now
00:36:38Araqand see what that does
00:40:39Araqnah, that works fine
00:40:50Araqmaybe it's the koch issue
00:42:12AraqJehan_: how is the scheduling done in your patch?
00:42:53AraqVarriount: nice. but all builders are offline
00:47:31*brson quit (Ping timeout: 272 seconds)
00:49:35*kniteli quit (Ping timeout: 265 seconds)
00:50:12dom96Varriount: Awesome. Nice job!
00:59:54*BlaXpirit-UA quit (Quit: Quit Konversation)
01:00:44*kniteli joined #nimrod
01:03:41*darkf joined #nimrod
01:05:09Jehan_Araq: Queue. Worker threads get a signal when there are tasks in the queue.
01:05:32Araqok
01:05:37Jehan_cpu load stuff is done via calling advice.
01:06:16Jehan_Depending on the result, fewer or more signals are being sent out.
01:06:42Araqhuh?
01:06:56Jehan_Araq: To wake up additional workers (or fewer).
01:07:56Araqwhy do you dislike the fixed size array for pending flowvars?
01:08:04AraqI think it's a pretty elegant solution
01:08:33Jehan_Not when there are a huge number of flowvars being GCed at once.
01:08:42Jehan_E.g. if you do a parmap type of thing.
01:09:05Jehan_Need to go to bed, though, we can talk more tomorrow.
01:09:14Jehan_Night!
01:09:17*Jehan_ quit (Quit: Leaving)
01:09:17Araqok, night
01:11:09NimBotnimrod-code/nimforum new_async c1cf06c Dominik Picheta [+0 ±1 -0]: Sidebar style adjustments.
01:11:09NimBotnimrod-code/nimforum new_async 77bcb14 Dominik Picheta [+0 ±3 -0]: Error handling for login in sidebar.
01:17:04Araqgood night
01:28:15*flaviu joined #nimrod
01:31:15superfuncnight
01:31:30flaviuWhat happens if I have a `method a(Foo)` in module 1, and a different `method a(Foo)` in module 2?
01:44:12*kniteli quit (Ping timeout: 265 seconds)
01:49:50*wayne joined #nimrod
01:53:51Varriountflaviu: Ambiguity error?
01:53:59VarriountHi wan
01:54:03Varriount*wayne
01:54:39*superfunc quit (Ping timeout: 272 seconds)
01:55:38flaviuThat sounds nasty, but I can't see a better solution.
01:56:38*kniteli joined #nimrod
02:00:16VarriountHello kniteli
02:02:30*kniteli quit (Ping timeout: 265 seconds)
02:13:57*kniteli joined #nimrod
02:33:40*saml_ joined #nimrod
02:51:25*darkf_ joined #nimrod
02:54:36*q66 quit (Quit: Leaving)
02:55:27*darkf quit (Ping timeout: 272 seconds)
03:00:56*AFKMorpork is now known as AMorpork
03:04:19*kniteli quit (Ping timeout: 272 seconds)
03:10:18*gokr_ quit (Ping timeout: 258 seconds)
03:16:06*kniteli joined #nimrod
03:28:52*darkf_ is now known as darkf
03:40:47*dapz joined #nimrod
03:49:11*brson joined #nimrod
03:51:19*xtagon joined #nimrod
04:20:57*kniteli quit (Ping timeout: 272 seconds)
04:22:56*brson_ joined #nimrod
04:23:51*dapz quit (Quit: My MacBook Pro has gone to sleep. ZZZzzz…)
04:24:10*brson quit (Ping timeout: 256 seconds)
04:27:08*dapz joined #nimrod
04:39:34*brson_ quit (Ping timeout: 255 seconds)
05:21:24*xtagon quit (Quit: Leaving)
05:43:14*flaviu quit (Ping timeout: 244 seconds)
05:44:53*fowl quit (Ping timeout: 264 seconds)
05:48:04*kniteli joined #nimrod
05:51:29*saml_ quit (Quit: Leaving)
05:52:29*kemet joined #nimrod
05:54:33*fowl joined #nimrod
05:56:59*q66[lap] quit (Read error: Connection reset by peer)
05:57:40*q66[lap] joined #nimrod
06:10:31*kniteli quit (Ping timeout: 265 seconds)
06:16:02*xenagi joined #nimrod
06:21:50*kniteli joined #nimrod
07:21:34*kniteli quit (Ping timeout: 265 seconds)
07:30:19*Matthias247 joined #nimrod
07:32:46*kniteli joined #nimrod
07:53:38*kemet quit (Quit: Instantbird 1.5 -- http://www.instantbird.com)
07:56:55*kniteli quit (Ping timeout: 272 seconds)
08:09:27*kniteli joined #nimrod
08:15:42*Demos quit (Read error: Connection reset by peer)
08:20:25*delian66 quit (Remote host closed the connection)
08:25:01*kniteli quit (Remote host closed the connection)
08:44:33*xenagi quit (Read error: Connection reset by peer)
08:45:23*johnsoft quit (Ping timeout: 240 seconds)
08:46:04*johnsoft joined #nimrod
09:07:50*darkf quit (Ping timeout: 265 seconds)
09:19:28*Sembei quit (Ping timeout: 255 seconds)
09:19:40*Pisuke joined #nimrod
09:41:46*BlaXpirit joined #nimrod
09:43:03*q66[lap] quit (Read error: Connection reset by peer)
09:43:46*q66[lap] joined #nimrod
10:12:32*sanjay joined #nimrod
10:13:52gokrVarriount: I am quite busy today, but tomorrow I can set up Linux slaves if you wish. The box is up and "waiting".
10:27:35*sanjay quit (Quit: Page closed)
10:47:53*uku joined #nimrod
10:48:43*dapz quit (Quit: Textual IRC Client: www.textualapp.com)
10:57:48*darkf joined #nimrod
11:03:38*uku quit (Ping timeout: 250 seconds)
11:28:30*darkf_ joined #nimrod
11:32:25*darkf quit (Ping timeout: 265 seconds)
11:43:55*uku joined #nimrod
11:54:43*darkf_ is now known as darkf
11:55:38*gokr_ joined #nimrod
12:37:51*def- quit (Quit: -)
12:38:30*def- joined #nimrod
12:40:24*BitPuffin joined #nimrod
12:50:48*q66 joined #nimrod
13:05:01*flaviu joined #nimrod
13:15:40*gokr_ quit (Remote host closed the connection)
13:15:55*gokr_ joined #nimrod
13:54:54*foodoo joined #nimrod
13:54:56Varriountgokr: Ok, thanks.
13:55:42*moritz left #nimrod (#nimrod)
13:57:59Varriountdom96, Araq: A webhook needs to be added to the Nimrod Github repo to point to http://178.62.143.63:9001
13:59:50foodoohm. Is it still "Nimrod" or will it be called "Nim" in the future?
14:00:34Varriountfoodoo: It's Nim, but it's going to be a while until I actually remember to call it that.
14:01:09Varriountfoodoo: For one thing, all my local... 'Nim' directories are named 'Nimrod'
14:07:19dom96Varriount: I don't have access to that unfortunately.
14:08:22dom96Araq: Should I rename nimrod-code to nim-code?
14:11:20dom96-+
14:11:26dom96 +
14:11:46dom96oops
14:12:02foodooYou ascii art brush dropped into the IRC window
14:23:40*BlaXpirit-UA joined #nimrod
14:24:52*BlaXpirit quit (Ping timeout: 245 seconds)
14:29:22*foodoo quit (Quit: leaving)
14:29:33rpagthe diff ascii brush
14:36:19*rpag quit (Ping timeout: 250 seconds)
14:42:36*darkf quit (Remote host closed the connection)
14:49:27*rpag joined #nimrod
15:11:17*BitPuffin quit (Ping timeout: 264 seconds)
15:11:52*BitPuffin joined #nimrod
15:14:40*kemet joined #nimrod
15:45:38*johnsoft quit (Ping timeout: 255 seconds)
15:47:50*johnsoft joined #nimrod
16:00:23*untitaker quit (Ping timeout: 250 seconds)
16:06:31*untitaker joined #nimrod
16:07:47*johnsoft quit (Ping timeout: 245 seconds)
16:08:40*johnsoft joined #nimrod
16:24:25*johnsoft quit (Ping timeout: 244 seconds)
16:24:36*johnsoft joined #nimrod
16:25:47*irrequietus joined #nimrod
17:22:39*xenagi joined #nimrod
17:22:57*xenagi left #nimrod (#nimrod)
17:40:06*perturbation joined #nimrod
17:40:46perturbationI forgot who it was that I was talking to on here about this... but I have to apologize about Fortran
17:41:03perturbationTurns out Fortran '03 (well, '95 onwards) is so much nicer than Fortran '77
17:41:56perturbationI mean, I'm still not going to use it for day-to-day stuff, but it's still much cleaner than I thought it was
17:43:42*whoot joined #nimrod
17:44:42whootHow can I "gc-safely" iterate over a seq?
17:45:36dom96whoot: Are you looking to iterate over it in multiple threads?
17:46:17whootyes
17:46:29whootI'm trying to use that from a handler of an async http server
17:46:39whootbut I reduced my problem to http://pastebin.com/dfWYAVez
17:47:04dom96Async does not use threads implicitly.
17:47:21dom96The reason that is not gc safe is because `s` is a global var.
17:48:00dom96It should work as long as you don't compile with --threads:on
17:48:14dom96if you want to use threads then declare s as a {.threadvar.}
17:48:20dom96http://build.nimrod-lang.org/docs/manual.html#threadvar-pragma
17:48:22whootyes, I was trying to spawn a thread per request. maybe it's absolute overkill
17:48:52whootmy bad, but I do want to be reading a global variable
17:49:02dom96If you're using async await at the same time then that sounds like a bad idea.
17:49:53dom96Async await may blow up completely with spawn.
17:50:08dom96If it doesn't then you're very lucky.
17:52:01whootok, disregard the whole threading stuff I mentioned
17:52:34*kniteli joined #nimrod
17:52:55whootsame problem happens if I try to iterate over a global seq from a request handler
17:53:46dom96are you compiling with --threads:on?
17:53:52whootyes, is that bad?
17:54:00*BitPuffin quit (Ping timeout: 250 seconds)
17:54:12dom96It's not bad. But it's the cause of this error.
17:54:32whootI'd like ideally to get the http server to use all processors, that's why I passed --threads:on
17:54:41*q66[lap] quit (Read error: Connection reset by peer)
17:54:44whootbut then I can't figure out how to iterate over this seq
17:54:46dom96Currently that's not supported.
17:55:23*q66[lap] joined #nimrod
17:58:32*kniteli quit (Ping timeout: 265 seconds)
18:00:56whootThanks for your help dom96. So if i understand correctly, it's unsupported because the async http server uses await and that does not play nice with spawn as of today. Did i understand correctly?
18:02:06dom96Yeah. Getting this to work is on our roadmap so we will hopefully have this functionality in the future.
18:02:33whootok, thanks once again
18:02:38dom96Async await is still in its infancy.
18:02:58whootso to have a multithreaded http server i shouldnt use the async one
18:03:17whootnvm I'll just figure it out myself
18:10:01*kniteli joined #nimrod
18:21:48*kniteli quit (Ping timeout: 265 seconds)
18:27:07*Jesin quit (Quit: Leaving)
18:29:30NimBotnimrod-code/nimforum new_async 114d322 Dominik Picheta [+0 ±4 -0]: Redesigned front page.... 4 more lines
18:32:28*wayne left #nimrod (#nimrod)
18:32:56*kniteli joined #nimrod
18:43:56flaviuwhoot: If you don't care about automatic memory management, you can do createShared(T, size)
18:44:20flaviuhttp://nimrod-lang.org/system.html#createShared,typedesc
18:49:41*kemet quit (Quit: Instantbird 1.5 -- http://www.instantbird.com)
18:49:54*kniteli quit (Ping timeout: 272 seconds)
18:50:47*perturbation quit (Quit: Leaving)
18:50:48*kemet joined #nimrod
18:54:07*Jesin joined #nimrod
19:00:11Araqdom96, dom96_ name it "nim-lang" then, makes more sense, right?
19:00:34dom96ok. But you'll need to fix the readme.
19:00:39Araqalso I don't know why you keep saying that async doesn't work with spawn
19:01:01Araqit should work
19:01:31dom96I never said it doesn't work
19:01:35dom96I said it's not supported.
19:02:01dom96I've never tested it so I don't know.
19:02:10Araqah ok
19:02:15*kniteli joined #nimrod
19:02:24Araqwell we should test it
19:03:09dom96I renamed it.
19:04:37Araqok, so what do I need to do?
19:06:27*BitPuffin joined #nimrod
19:07:40gokr_whoot: Check my article about spawningserver
19:08:38gokr_spawn uses a threadpool btw
19:10:12Varriountgokr_: The build slave shouldn't need any special requirements; The only things that are required are the buildbot software, and git.
19:10:51VarriountYou'll need a password and an address to connect to - I'll PM you the information.
19:11:00gokr_sure
19:11:02*kniteli quit (Ping timeout: 265 seconds)
19:17:10dom96Araq: change all http://github.com/nim-code/* urls to http://github.com/nim-lang/*
19:17:23Araqwhere?
19:17:36*saml_ joined #nimrod
19:18:13*lbmn joined #nimrod
19:18:33Araqhi lbmn welcome
19:18:37Araqdom96: http://forum.nimrod-lang.org/t/622
19:19:50gokr_nice!!
19:20:07dom96well, I was just going to redirect to google.
19:22:20VarriountAraq: Have you added the web hook for the repository?
19:22:29*kniteli joined #nimrod
19:24:29flaviuVarriount: You can actually even get by withou git
19:25:06*tillzy joined #nimrod
19:25:09Varriountflaviu: Not with the buildbots. The master configuration uses Git.
19:26:15AraqVarriount: "just the push event" or "everything" or "individual events"?
19:26:24VarriountJust the push event.
19:26:57Araq"Okay, that hook was successfully created. We sent a ping payload to test it out! Read more about it at https://developer.github.com/webhooks/#ping-event."
19:27:40whootthanks gokr_
19:28:01*saml_ quit (Ping timeout: 265 seconds)
19:28:09VarriountAraq: This ping will fail - the bot will only respond to push events.
19:30:16AraqVarriount: ok, well, that shouldn't matter.
19:32:07lbmnIs there an alternative json library binding for nimrod?
19:32:33lbmn(Did you catch the decoding benchmark results I've mentioned last time?)
19:35:09Varriountlbmn: Not really. I mean, if there's a fast C or C++ json library out there, it can be wrapped, but I don't know of any currently existing alternative wrapper.
19:35:45VarriountAside from that, I would really rather have the pure-nimrod implementation upgraded.
19:37:00VarriountAraq: On a related note, I wasn't able to implement the string assignment optimization - the compiler kept crashing on bootstrap.
19:40:48lbmnWith all optimizations turned on, Nimrod's default json decoding speed is behind many scripting languages, which for some cases would mean the whole program would be slower.
19:41:10lbmnI realize that some json decoders do more than others, but still...
19:41:46*saml_ joined #nimrod
19:41:46flaviulbmn: Can you post benchmark programs?
19:42:01lbmnToo ugly for github right now...
19:42:19flaviuprivate gist?
19:42:27VarriountDoes the embedded profiler still work?
19:42:55flaviulinux perf is really excellent, I'll be testing it with that
19:43:18flaviuYou can't beat instruction-level data
19:43:55VarriountAraq: Now that the main build configuration is done, I'll be working on the manual builds, and then the web interface.
19:44:05lbmnJust giving a bit of feedback. Switching json libraries would be a low-effort way of getting a noticeable performance boost.
19:46:15lbmnThe program is as simple as it gets - http://pastebin.com/nBg05sam
19:47:15*q66[lap] quit (Read error: Connection reset by peer)
19:48:50*saml_ quit (Ping timeout: 250 seconds)
19:49:31*q66[lap] joined #nimrod
19:50:06lbmnluajit version that runs in less than half the time (1.26s, vs nimrod's 2.95s, or 12.50s without optimizations) - http://pastebin.com/CUfxauQx
19:51:09Varriountlbmn: Yes, switching libraries would be a low effort way, but then users would be required to compile the luajit library.
19:51:41flaviuVarriount: There are other libraries, including some specificly designed for embedding
19:51:50flaviuSome are likely faster than luajit
19:52:42VarriountHrmf. I'd still rather port a library over. There should be some things we provide that shouldn't rely on 3rd party libraries.
19:53:17lbmnI mean including nimrod bindings for a fast C json library like jsmn, ujson, yajl, etc.
19:53:38flaviuVarriount: even porting one isn't going to be too good for javascript.
19:53:55flaviuWhen you should the browser's C version
19:55:00*kniteli quit (Ping timeout: 244 seconds)
19:55:01*kemet quit (Quit: Instantbird 1.5 -- http://www.instantbird.com)
19:56:39lbmn(Node.JS runs that benchmark in 1.90 with require, 1.93 with JSON.parse.)
20:00:35lbmnStrange enough, all "C-alternative" compiled languages lag behind, not just nimrod: rust 3.2s, go 3.63.
20:01:53*saml_ joined #nimrod
20:03:13*askatasuna joined #nimrod
20:04:16gokrflaviu: The cjson library that the luajit test uses is actually a pure C library.
20:04:23gokrIts the one I pointed out yesterday.
20:05:15gokrSince LuaJIT calls into C just as fast as C does, result is ... fast.
20:06:39*kniteli joined #nimrod
20:06:53*kemet joined #nimrod
20:16:37Araqgokr: iirc luajit calls into C faster as C does because it can inline even more
20:16:45Araq*faster than
20:16:52gokrEven that yeah.
20:17:14flaviuis v8 somehow cheating?
20:17:35flaviuIt spends all the time fetching the file, parsing the json is instant
20:18:06Varriountlbmn: At least nimrod is faster than rust.
20:19:40gokrI peeked at the source of ultrajson on github, hehe, lots of memcpy etc. Looks heavily optimized.
20:20:09gokrhttps://github.com/esnme/ultrajson/tree/master/lib
20:20:12Varriountgokr: Was that sarcasm?
20:20:43flaviuVarriount: That sort of argument is pretty terrible. "This isn't the slowest, so use it"
20:22:09Araqflaviu: that's a misrepresentation of Varriount's argument.
20:22:34Araqso where can I find this json test file?
20:22:45Araqbut I think I know why it's slow in Nim anyway
20:22:54flaviuAraq: I don't know about his, but I'm using https://github.com/zemirco/sf-city-lots-json/blob/master/citylots.json
20:23:12gokrVarriount: No... and I kinda misread the code anyway. But its meant to be fast anyway.
20:23:12flaviu190MB
20:23:49Araqok, just as I suspected
20:24:10Araqlol I don't even need profilers anymore :P
20:24:45*Jehan_ joined #nimrod
20:25:22Araqhi Jehan_. I really like my implementation better. but can you give some hints as to why everything in it is wrong?
20:26:00*kniteli quit (Ping timeout: 244 seconds)
20:26:16Jehan_Araq: Umm, I'm not saying that "everything is wrong".
20:26:45Araqwell when I read your commit message, that surely reads like "bug here and here and here ..."
20:26:50Jehan_Except for the attempt to load balance at the userland level, which I think is just a waste of CPU cycles.
20:27:11Jehan_Yeah, race conditions, such as the one that got you stuck in a deadlock.
20:27:24Araqwell but that's exactly the feature that gave gokr good results
20:28:08Araqit's also similar to what .NET's thread pool does
20:28:43lbmnOne C JSON libraries benchmark - http://lionet.livejournal.com/118853.html (In Russian, scroll down to results.)
20:29:49Jehan_The biggest problem I see with the code is that it uses a mix of techniques that are really, really difficult to validate as being correct.
20:30:29*tillzy quit (Quit: tillzy)
20:30:41Jehan_The barrier code, for example, is still not correct, as far as I can tell.
20:31:06Jehan_It happens to not break on x86 processors, but I wouldn't bet on it for other types of processors.
20:31:34lbmnRust version in that benchmark is incomplete, I gave up before I figured out how to print the count (which shouldn't affect it by much). Compiled with `rustc --opt-level 3`. http://pastebin.com/pK4KFQEv
20:34:09lbmnI'm just saying, people do a cost-benefit analysis in their head before switching from their ScriptLang to Nimrod. There has to be a performance boost, and usually there is, but with a JSON-heavy program there won't be.
20:34:55Jehan_Then there's the problem with nimSpawn. It's not that I actually think it has a bug (at least not on TSO processors), there's just no way I can validate its correctness, and hence I don't trust it, because you can't reliably test parallel code (assuming it isn't specifically designed for being testable).
20:35:00*tillzy joined #nimrod
20:35:14flaviulbmn: You're using an old version of rust
20:36:36Jehan_People stick with traditional monitor semantics for most stuff because the semantics are extremely well understood and can be reasoned about. Doing dirty reads and using lower-level primitives has to be done with an extreme level of care if you want to be able to trust the code.
20:36:56AraqJehan_: ok, please tell me *why* the barrier is still not correct
20:37:11lbmnThe nightly one was FUBAR.
20:37:29*kniteli joined #nimrod
20:37:34Araqlbmn: there is no argument here. one way or the other we'll improve json parsing.
20:37:45gokrAraq: Curious what you "knew" was the problem in the JSON lib :)
20:37:55Jehan_Araq: Because on some architectures, barrierLeave may not see the most recent value of b.entered
20:38:42flaviulbmn: .12 works though
20:39:01Jehan_To be precise, it's again an "I can't be sure" thing, because it may still accidentally turn out to be correct. But I cannot trust the code.
20:39:13AraqJehan_: there is a comment in barrierEnter that explains why I think there is no "fence" necessary
20:39:14*Jehan_ quit (Quit: Leaving)
20:39:36*Jehan_ joined #nimrod
20:39:53lbmnI don't like Rust anyway. Nimrod has the best syntax of any language on the system's half of Ousterhout's tennis court...
20:40:11Jehan_Araq: I know. The problem is that ordering memory accesses is not enough.
20:40:24Jehan_Just because you have a fence somewhere doesn't mean stuff gets written to memory.
20:40:31Jehan_It means that reads/write don't get reordered.
20:40:35Araqer ... well
20:40:43Araqthat's what I thought it does
20:41:24Jehan_Fences basically mean that reads/writes before the fence don't get swapped around with reads/writes after.
20:41:36AraqI see
20:41:40Jehan_There's no generic "flush all stuff to memory" instruction on all processors.
20:41:45Jehan_Now, here's the thing.
20:42:16gokrBtw, I don't want to bother Araq with questions on the exception system in Nim - anyone care to comment on it - say contrasted with the Java model?
20:42:41Jehan_It may still work because stuff like thread creation creates its own barriers.
20:43:00Jehan_And so forces events to be sequentialized at a global level so that it still works.
20:43:58Araqwell yes, but on the other hand, I tested it and actually *debugged* it.
20:44:09Araqso it certainly didn't always work
20:44:25Jehan_Araq: Yeah. The thing is, you tested it on an Intel or AMD processor.
20:44:43Jehan_They're TSO processors, which is nearly as good as "always sequentially consistent".
20:45:02wanlbmn: jsmn looks awesome. On its page: "no dynamic memory allocation". They must be spliting the original string, no wonder it's the fastest. No copy! No alloc! I'm not even sure Nim can do that (certainly not with a json module backed by a hash table using ref).
20:45:09Jehan_The downside is that a full fence instruction on such a processor is really, really expensive.
20:45:34Jehan_Pipeline stall while waiting for data written to memory in the worst case.
20:46:36Jehan_On the other hand, pure read barriers and pure write barriers are nops.
20:46:51Jehan_And overall it's much saner to deal with than, say, ARM. :)
20:49:22Jehan_In the general case, however, if you want to make sure that a write in one thread becomes visible to another thread, then the write must be succeeded by a handshake operation, and the handshake operation in the other thread must precede the read that that thread does.
20:49:34Jehan_This is what condition variables and semaphores accomplish.
20:50:15Araqwan: Nim can easily do it. now ok, the stdlib doesn't support it, but then C barely even has a stdlib. so you end up with C-like Nim code. Which still has advantages over C.
20:52:34AraqJehan_: I'm not even sure I like semaphores better than CAS+races. The thing is ... with a semaphore I have to make liveness guarantees all the time. and I dunno how you can reason about *that*.
20:52:43gokrGood page on jsmn: http://zserge.com/jsmn.html
20:52:59gokrSo it simply points to where the data starts/ends in the original string.
20:53:24Jehan_What liveness guarantees do you mean?
20:53:58Araqwhenever I call 'await' there is no timing involved. I need to ensure 'signal' really will be called eventually.
20:54:33Jehan_That's honestly why locks + condition variables are often the better solution.
20:54:47lbmnIt would be awesome to have many alternatives for performance-critical libraries like json, bonus points if they share the same method names / etc so switching between them is easy.
20:55:38lbmnLike python's anyjson (although pypy json beats CPython's best C libraries for some reason).
20:55:58wanAraq: I'm probably unexperienced on that side of Nim, I had troubles before. I can get a pointer with addr(a), but doing something with it, especially like casting a ptr to an array and using it... I must try again.
20:56:11AraqJehan_: well yeah but I came up with the CondVar because locks+condition variables have their own problems, at least for me.
20:56:14lbmnThen people can switch based on circumstances, like file size, memory limits, number of CPU cores, etc.
20:56:26Jehan_lock; while not predicate: wait; unlock matches with lock; update predicate; signal; unlock is a pretty universal mechanism how two threads can agree on pretty much arbitrarily complex predicates while maintaining thread safety.
20:56:27*Sembei joined #nimrod
20:57:07flaviuwan: You can't cast a pointer to an array, a pointer is referencing a location memory, while an array is directly memory
20:57:34flaviuArrays are c-style, not java style
20:57:54Jehan_Araq: That's essentially a semaphore, and while semaphores have their uses, they also provide somewhat weaker guarantees.
20:58:16*Pisuke quit (Ping timeout: 258 seconds)
20:58:41*johnsoft quit (Ping timeout: 260 seconds)
20:58:52Araqwan: well I can do it and I guess we need better docs, as usual
20:58:56*johnsoft joined #nimrod
20:59:11wanflaviu: so you can't do `somevar[count] = ...` if somevar is a ptr? That's what I was trying to do at the time, if I recall correctly.
20:59:40flaviua ptr array[T, ...]?
20:59:41Araqwan: use a ptr to an array to do that. no big deal at all.
20:59:58wanAs I said, I must try again.
21:00:04Jehan_The biggest limitations are that (1) you have no atomicity guarantees for the state you're operating on and (2) the number of signals must match the number of waits (which can lead to additional bookkeeping).
21:00:57gokrwan: jsmn is on the other hand only a tokenizer. Doesn't do anything more.
21:06:06AraqJehan_: wie auch immer. I don't want to merge your PR because it feels like losing all my work and I don't want to dismiss your PR because that would lose your work... Can't we have some -d:paranoid flag for weak CPU architectures?
21:07:04Jehan_Araq: First, don't worry about my work. That was just a couple of hours of coding, and it was faster than writing it up as text.
21:07:13wangokr: what else should a json parser do?
21:07:37gokrI mean, it doesn't do number parsing for example. And not char escape codes etc.
21:07:57gokrIIUIC
21:09:15wanI've checked how jsmn "no alloc" works: you need to give it space for tokens yourself (either on the stack or heap, beforehand). So, no arbitrary number of nodes, but it's still cool
21:09:23*johnsoft quit (Ping timeout: 258 seconds)
21:09:41Jehan_But, the whole current approach worries me in that it seems to be too much concerned with microoptimization (without actual evidence that there's a payoff) rather than putting correctness first.
21:10:18*johnsoft joined #nimrod
21:10:20wangokr: there is a test with escape stuff, I think it covers it
21:11:06wanbut yeah, number parsing is important
21:11:56gokrI just read some discussion on it.
21:13:00AraqJehan_: well sure. on the other hand the barrier does pay off according to you. and avoiding allocShared has its advantages too.
21:14:13AraqI mean, currently spawn itself does no allocation whatsoever (I think) and so ensures bounded memory usage. which some domains really require.
21:15:36Jehan_Umm, if there's a performance difference, it's in the low single digit percentage range, from what I've been able to tell (may be different on other OSs, I don't know).
21:16:07Jehan_To improve performance, I'd mostly avoid recreating the semaphore each time.
21:16:47Jehan_You could cache a thread-local instance and only create a new one in the rare case that you have nested parallel statements.
21:17:20Jehan_The biggest problem there IMO is still the necessary synchronization in nimArgsPassingDone and the serialization bottleneck that entails, though.
21:19:43Araqok, but yours is also not exception safe and so would require an implicit try-finally which we don't generate yet, adding to its cost
21:21:57Araqapart from that I don't really use micro optimizations for the sake of it. I find the 'cas' in the code much easier to reason about than what I had before.
21:22:32Jehan_What precisely about it is not exception safe that isn't already a problem in the existing code?
21:23:27Araqthe cond var stuff that requires freeing only happens in 'closeBarrier', it's not spread out.
21:24:13Araqso we can get away without any 'try-finally', but even if we wouldn't, it would only happen if b.left != b.entered
21:24:22Jehan_I see.
21:25:18Jehan_That can be fixed, though.
21:25:50*kemet quit (Quit: Instantbird 1.5 -- http://www.instantbird.com)
21:26:12Jehan_The primary problem is that the check for b.left == b.entered is not safe.
21:30:17wanlbmn: for me, using -d:release, with the 190MB file, nim takes 4.9s and luajit takes 4.2s. I think it's pretty good. Have you tried forgetting all your switches and just using -d:release?
21:34:03wanIt does take a lot more memory though, I think luajit is around 600-700MB, and nim goes to 1150MB for that same file of 181MB
21:37:29*kniteli quit (Ping timeout: 265 seconds)
21:38:21Jehan_Araq: Somewhat unrelated, is there an atThreadExit/atThreadInit functionality?
21:39:10Jehan_I couldn't find any, but that obviously doesn't mean it's not there.
21:41:32gokrwan: Then perhaps we can simply tune what we have for now.
21:41:48*Trustable joined #nimrod
21:42:54*whoot quit (Quit: Leaving)
21:46:37*irrequietus quit ()
21:48:38Jehan_Anyhow, have to get up early tomorrow, good night. :)
21:48:39*Jehan_ quit (Quit: Leaving)
21:48:54*kniteli joined #nimrod
21:55:51lbmnMy Facebook Page cover image now mentions Nimrod... https://www.facebook.com/copyfree.org (To be updated soon...)
21:56:20lbmnGotta disconnect. If you need me, msg me on Facebook or [email protected]...
21:56:41*lbmn quit (Quit: Leaving)
21:57:03gokrAnyone here using gitlab btw?
22:06:00*kniteli quit (Ping timeout: 265 seconds)
22:08:25gokrVarriount: Can you give me just an itsy bitsy more info on what I should do to set up a slave for you? :)
22:11:53Varriountgokr: Well, you need to install the buildot slave package ('pip install buildbot slave'), then create a slave ('buildslave create-slave <slave_dir> 178.62.143.63:9989 linux-x32-slave-1 <password>')
22:12:25Varriountthen start the slave, ('buildslave start <slave_dir>')
22:12:30flaviuHas anyone managed to get nim to parse the file?
22:12:44Varriountflaviu: Parse what file?
22:12:50flaviuthe json
22:12:56gokrOk, and... this is a 64 bit Linux, but I presume the build will use the -m32 flags etc?
22:13:30Varriountgokr: The two linux build slaves are named 'linux-x32-slave-1' and 'linux-x64-slave-1'
22:14:07gokrWhich version are you on? 0.8.9?
22:14:33VarriountYep.
22:15:35Varriountgokr: Other than that, you'll need git for checking out the repositories, and sqlite for the testers.
22:15:47gokryeah
22:16:14gokrWonder if I should do this in docker or not. Will try first "regularly".
22:17:02VarriountAfter that, the slaves should be relatively low-maintenance. All the build steps are given by the master, and any auxilary scripts used locally are downloaded via git.
22:17:21*kniteli joined #nimrod
22:17:47Varriountgokr: If you can, have the build slaves restart automatically if they shutdown due to being disconnected from the build master.
22:18:15gokrok
22:19:45Varriountgokr: Thanks for the support, it's greatly appreciated.
22:20:11*Matthias247 quit (Read error: Connection reset by peer)
22:20:37gokrWe are going to use Nim, so we should "chip in".
22:20:58wanflaviu: I tried the nim program that lbmn did, run fine with the big file
22:21:11Varriountgokr: You can see builder status on http://178.62.143.63:8010/waterfall
22:21:20flaviuI'm getting a syntax error on line 1
22:21:27gokrVarriount: right
22:25:14wanflaviu: did you paste it from the RAW pastebin area? The first line is fine, it's just import json, strutils
22:25:41flaviuNo, a syntax error in the json file. Really weird, maybe I should pull the latest nim
22:26:48wanI wgetted the file from the raw github link. Maybe yours stopped half-time while downloading?
22:26:51*kniteli quit (Ping timeout: 265 seconds)
22:28:32flaviuI did exactly the same
22:29:00flaviuYep, the file looks good to me.
22:29:26wan158346af5a90253d8b4390bd671eb5c5 citylots.json
22:29:33wan(md5sum)
22:30:32flaviuIt's the same file, although you should migrate to sha256 asap
22:34:17flaviuinput(1, 2) Error: string literal as key expected expected
22:34:27flaviuAnyway, I'll figure it out later
22:36:56wanI'm using Nim Compiler Version 0.10.1 (2014-11-08) [Linux: amd64] if you want to know.
22:38:19*kniteli joined #nimrod
22:41:46Varriountgokr: I know why the linux builder failed - it's a bug in the master config.
22:41:53gokrah cool
22:41:57gokrIt tried at least :)
22:42:15Varriountgokr: Araq doesn't want the execute bit set for the build.sh script. :/
22:42:38gokrAh, yeah, I can see it in the log
22:43:01gokrI will create the 64 too.
22:44:03gokrVarriount: Same pass for 64?
22:45:34gokrSo we will be building the Linux stuff in Las Vegas. :)
22:45:36*Trustable quit (Quit: Leaving)
22:45:47AraqVarriount: I don't mind the exec bit
22:45:59Araqbut I think it's more robust to not rely on it
22:46:02Varriountgokr: Yes.. although I don't have experience with setting up dual builds on linux.
22:46:19Araqif it pains you go ahead and make it exec.
22:46:21gokr"dual builds"?
22:46:30Varriountgokr: 32 and 64 bit builds.
22:46:34Araqyour productivity is much more important
22:46:52gokrBut... the build scripts are different?
22:46:56gokrI hope :)
22:47:02VarriountUh, not really.
22:47:06gokrOh.
22:47:18Varriountgokr: https://github.com/nim-lang/nim-buildbot/blob/master/build_steps.py
22:47:22dom96Why can't you simply execute `sh build.sh`?
22:47:23gokrOk, checking
22:47:32Varriountdom96: That's what I just changed it to.
22:49:37gokrHmmm, that log looks a bit oddish. I think I will start the slave differently.
22:55:20VarriountI'm hoping that I can modify the build steps and the build configuration enough to generate a comparitive view of the test results.
22:55:31VarriountRight now the results just print to stdout.
22:55:44AraqVarriount: the tester already supports that
22:55:48AraqI hope you know
22:55:52gokrVarriount: csources built now last time.
22:56:17VarriountAraq: The tester supports html output and json output, but only for the platform it's been tested on.
22:56:33VarriountYou can't get a comparitive view of two different platforms.
22:56:36gokrVarriount: Then it failed on "nimrod c koch.nim". Should it be nimrod?
22:57:22AraqVarriount: maybe but it's easy to add support for that, the database model has the concept of a platform
22:57:54gokrVarriount: Force again, I started it differently.
22:59:45gokrAnd... the 64 slave failed: unauthorized login; check slave name and password
23:00:12VarriountI'll PM you the password and name again, it's possible I made a mistake.
23:00:48gokrThe name for the slave seems correct.
23:00:56gokrSo perhaps a different pass.
23:01:01gokrAh, that's different :)
23:02:23gokrNow it started fine.
23:02:48flaviuVarriount: I have a arm v5 device, can you PM me the password?
23:03:25Varriountflaviu: Uh, I haven't created a slave account for that platform yet.
23:03:50Varriountflaviu: I'll pm you a set of slave credentials, just give me a bit to actually set the slave account up.
23:04:10flaviuoh, I thought it was global. That's fine
23:05:02gokrVarriount: You now... you are building "nim" with csources.
23:05:07gokrVarriount: Not "nimrod"
23:06:15Varriountgokr: On windows, the binary name generated by csources is nimrod. I guess it's different on linux
23:06:30gokrYou can tell if you look at the log from the step before.
23:06:54VarriountRegardless, the normalization step should take care of that.
23:07:47*brson joined #nimrod
23:11:28*Demos joined #nimrod
23:15:38*uku quit (Ping timeout: 255 seconds)
23:18:24*uku joined #nimrod
23:21:34*tillzy quit (Quit: tillzy)
23:22:51*kniteli quit (Ping timeout: 265 seconds)
23:25:13Varriountgokr: Can you check what binaries are present in the slaves build directory?
23:25:21gokrmmm
23:26:35gokrIn linux32, in bin - there is both a "nim" and a "nimrod" - both same size, but only "nim" is executable.
23:26:47VarriountAh, that would be it..
23:27:08Varriountgokr: Apparently python's copyfile() doesn't copy the executable bit..
23:27:17gokryeah
23:28:11flaviuVarriount: Sorry to bother you again, but I may have rudely killed my slave, causing problems. Can you do something about "rejecting duplicate slave"
23:28:54gokrSo... is it you who are copying "nim" to "nimrod" in your build script?
23:29:05Varriountflaviu: Make sure no zombie processes are running. I can't do anything from my side other than restarting the master.
23:29:39gokrAlso, docker seemingly doesn't do 32 bits on 64 yet. So for a "true" 32 bit Linux, I think we need vagrant.
23:29:55flaviuVarriount: You're right, I messed up the auto-restart functionallity
23:33:01gokrflaviu: What autorestart thingy do you use?
23:33:08flaviugokr: systemd
23:33:12gokrok
23:33:27flaviuI'll gist my service file once I get everything working
23:33:58Varriountflaviu: What distro are you using?
23:34:03flaviuArch linux
23:34:24*kniteli joined #nimrod
23:34:26VarriountAh. So Arch doesn't have a buildbot package/doesn't set up a buildmaster user for you?
23:35:33flaviuIt doesn't have an official or AUR package for buildbot. I could make an AUR package, but meh, too much work.
23:37:14wangokr: for me (on linux), csources make a nimrod executable, not nim
23:37:28Varriountflaviu: What user are you running the build slave under?
23:37:46flaviuI made a special user for it, but I need to lock it down a bit more
23:38:31Varriountwan: I'm aware of that. There's a script I run which creates a nimrod/nim executable based on whether a nim/nimrod executable already exists.
23:41:34*bjz quit (Ping timeout: 258 seconds)
23:43:29*wan quit (Quit: WeeChat 1.0.1)
23:44:56Varriountgokr: Ok, something must be off.
23:45:12gokrmm?
23:45:51gokrAh, you changed it to nim
23:45:56VarriountOh, wait, nevermind
23:46:49gokrNow none of them are executable :)
23:47:02VarriountYeah, I realized that.
23:47:06Varriount>_<
23:48:25flaviuhttps://gist.github.com/flaviut/9ddfcb94ebe6161b3bc6
23:48:53gokrflaviu: cool
23:49:19flaviuSeems to be working for me, but given the speed that this device moves, it'll be a while until I even finish bootstrapping
23:49:40Varriountflaviu: What device would this be?
23:49:48gokrand... exec-buildbot script?
23:49:55flaviuPogoplug series 4
23:50:09flaviugokr: Just a wrapper to set up my virtualenv
23:50:15gokrok
23:50:19flaviuLet me fix that
23:50:23Varriountgokr: *gasp*, look at the waterfall page.
23:50:47gokr:)
23:51:39flaviuVarriount: mac doesn't support x32
23:52:00flaviuoh, wait
23:52:06Varriountflaviu: Uh, yes it does.
23:52:28flaviuAll their CPUs support 64, but it looks like 32 is supported for some random reason
23:54:25flaviutsk, tsk Varriount. You need to edit buildbot/info/* :P
23:54:31flaviuhttp://178.62.143.63:8010/buildslaves
23:57:00Varriountflaviu: Huh?
23:57:03AraqVarriount: does that github hook work?
23:57:15flaviuVarriount: Your Name Here <[email protected]>
23:57:34VarriountAraq: I think so. It should work...
23:57:57flaviuVarriount: And your buildbot description is "Please put a description of this build host here"
23:58:15Varriountflaviu: Details, details...
23:58:41flaviuYes, but it looks sloppy if you don't pay attention to them