00:01:11 | * | Trixar_za quit (Ping timeout: 265 seconds) |
00:04:16 | * | Matthias247 quit (Read error: Connection reset by peer) |
00:04:23 | * | uber joined #nimrod |
00:04:23 | * | uber quit (Max SendQ exceeded) |
00:04:25 | * | Trixar_za joined #nimrod |
00:04:53 | * | uber joined #nimrod |
00:17:44 | * | uku quit (Ping timeout: 255 seconds) |
00:23:17 | Varriount | dom96: ping |
00:23:55 | dom96 | Varriount: Please just ask me instead of pinging. |
00:24:46 | Varriount | Ok. dom96: Does the VPN have any firewall or file which needs to be altered to let traffic through to a particular port? |
00:25:30 | Varriount | I have the builder listening on all interfaces, at port 8010, yet when I connect to nim-lang.org:8010, I get a 'ERR_CONNECTION_REFUSED' status in my web browser. |
00:25:54 | dom96 | nim-lang.org does not point to that IP |
00:26:20 | * | kniteli joined #nimrod |
00:27:11 | dom96 | No, you do not need to do anything for that port to work. |
00:29:21 | Araq | Varriount: any luck with the "nim picks the wrong stdlib" issue? |
00:31:24 | Varriount | Araq: I've been working on the buildbot, so no. |
00:31:28 | * | BitPuffin quit (Ping timeout: 244 seconds) |
00:33:56 | Varriount | Araq, dom96: http://178.62.143.63:8010/waterfall |
00:34:17 | Jehan_ | Araq: Where does the issue show up, is it specific to some OS or a general thing? |
00:35:21 | Araq | Jehan_: I had it on windows when testing something else |
00:35:33 | Jehan_ | Hmm. |
00:35:42 | Araq | and from lots of reports here in #nimrod I suspect it's a general issue |
00:35:54 | Varriount | It's never effected me. |
00:35:57 | Jehan_ | For what it's worth, when I ran ./koch test, it picked the nim from my PATH rather than the one in bin |
00:35:59 | Varriount | *affected |
00:36:16 | Jehan_ | But I think that's something different. |
00:36:32 | Araq | well I dunno. I'm installing 0.9.2 now |
00:36:38 | Araq | and see what that does |
00:40:39 | Araq | nah, that works fine |
00:40:50 | Araq | maybe it's the koch issue |
00:42:12 | Araq | Jehan_: how is the scheduling done in your patch? |
00:42:53 | Araq | Varriount: nice. but all builders are offline |
00:47:31 | * | brson quit (Ping timeout: 272 seconds) |
00:49:35 | * | kniteli quit (Ping timeout: 265 seconds) |
00:50:12 | dom96 | Varriount: Awesome. Nice job! |
00:59:54 | * | BlaXpirit-UA quit (Quit: Quit Konversation) |
01:00:44 | * | kniteli joined #nimrod |
01:03:41 | * | darkf joined #nimrod |
01:05:09 | Jehan_ | Araq: Queue. Worker threads get a signal when there are tasks in the queue. |
01:05:32 | Araq | ok |
01:05:37 | Jehan_ | cpu load stuff is done via calling advice. |
01:06:16 | Jehan_ | Depending on the result, fewer or more signals are being sent out. |
01:06:42 | Araq | huh? |
01:06:56 | Jehan_ | Araq: To wake up additional workers (or fewer). |
01:07:56 | Araq | why do you dislike the fixed size array for pending flowvars? |
01:08:04 | Araq | I think it's a pretty elegant solution |
01:08:33 | Jehan_ | Not when there are a huge number of flowvars being GCed at once. |
01:08:42 | Jehan_ | E.g. if you do a parmap type of thing. |
01:09:05 | Jehan_ | Need to go to bed, though, we can talk more tomorrow. |
01:09:14 | Jehan_ | Night! |
01:09:17 | * | Jehan_ quit (Quit: Leaving) |
01:09:17 | Araq | ok, night |
01:11:09 | NimBot | nimrod-code/nimforum new_async c1cf06c Dominik Picheta [+0 ±1 -0]: Sidebar style adjustments. |
01:11:09 | NimBot | nimrod-code/nimforum new_async 77bcb14 Dominik Picheta [+0 ±3 -0]: Error handling for login in sidebar. |
01:17:04 | Araq | good night |
01:28:15 | * | flaviu joined #nimrod |
01:31:15 | superfunc | night |
01:31:30 | flaviu | What happens if I have a `method a(Foo)` in module 1, and a different `method a(Foo)` in module 2? |
01:44:12 | * | kniteli quit (Ping timeout: 265 seconds) |
01:49:50 | * | wayne joined #nimrod |
01:53:51 | Varriount | flaviu: Ambiguity error? |
01:53:59 | Varriount | Hi wan |
01:54:03 | Varriount | *wayne |
01:54:39 | * | superfunc quit (Ping timeout: 272 seconds) |
01:55:38 | flaviu | That sounds nasty, but I can't see a better solution. |
01:56:38 | * | kniteli joined #nimrod |
02:00:16 | Varriount | Hello kniteli |
02:02:30 | * | kniteli quit (Ping timeout: 265 seconds) |
02:13:57 | * | kniteli joined #nimrod |
02:33:40 | * | saml_ joined #nimrod |
02:51:25 | * | darkf_ joined #nimrod |
02:54:36 | * | q66 quit (Quit: Leaving) |
02:55:27 | * | darkf quit (Ping timeout: 272 seconds) |
03:00:56 | * | AFKMorpork is now known as AMorpork |
03:04:19 | * | kniteli quit (Ping timeout: 272 seconds) |
03:10:18 | * | gokr_ quit (Ping timeout: 258 seconds) |
03:16:06 | * | kniteli joined #nimrod |
03:28:52 | * | darkf_ is now known as darkf |
03:40:47 | * | dapz joined #nimrod |
03:49:11 | * | brson joined #nimrod |
03:51:19 | * | xtagon joined #nimrod |
04:20:57 | * | kniteli quit (Ping timeout: 272 seconds) |
04:22:56 | * | brson_ joined #nimrod |
04:23:51 | * | dapz quit (Quit: My MacBook Pro has gone to sleep. ZZZzzz…) |
04:24:10 | * | brson quit (Ping timeout: 256 seconds) |
04:27:08 | * | dapz joined #nimrod |
04:39:34 | * | brson_ quit (Ping timeout: 255 seconds) |
05:21:24 | * | xtagon quit (Quit: Leaving) |
05:43:14 | * | flaviu quit (Ping timeout: 244 seconds) |
05:44:53 | * | fowl quit (Ping timeout: 264 seconds) |
05:48:04 | * | kniteli joined #nimrod |
05:51:29 | * | saml_ quit (Quit: Leaving) |
05:52:29 | * | kemet joined #nimrod |
05:54:33 | * | fowl joined #nimrod |
05:56:59 | * | q66[lap] quit (Read error: Connection reset by peer) |
05:57:40 | * | q66[lap] joined #nimrod |
06:10:31 | * | kniteli quit (Ping timeout: 265 seconds) |
06:16:02 | * | xenagi joined #nimrod |
06:21:50 | * | kniteli joined #nimrod |
07:21:34 | * | kniteli quit (Ping timeout: 265 seconds) |
07:30:19 | * | Matthias247 joined #nimrod |
07:32:46 | * | kniteli joined #nimrod |
07:53:38 | * | kemet quit (Quit: Instantbird 1.5 -- http://www.instantbird.com) |
07:56:55 | * | kniteli quit (Ping timeout: 272 seconds) |
08:09:27 | * | kniteli joined #nimrod |
08:15:42 | * | Demos quit (Read error: Connection reset by peer) |
08:20:25 | * | delian66 quit (Remote host closed the connection) |
08:25:01 | * | kniteli quit (Remote host closed the connection) |
08:44:33 | * | xenagi quit (Read error: Connection reset by peer) |
08:45:23 | * | johnsoft quit (Ping timeout: 240 seconds) |
08:46:04 | * | johnsoft joined #nimrod |
09:07:50 | * | darkf quit (Ping timeout: 265 seconds) |
09:19:28 | * | Sembei quit (Ping timeout: 255 seconds) |
09:19:40 | * | Pisuke joined #nimrod |
09:41:46 | * | BlaXpirit joined #nimrod |
09:43:03 | * | q66[lap] quit (Read error: Connection reset by peer) |
09:43:46 | * | q66[lap] joined #nimrod |
10:12:32 | * | sanjay joined #nimrod |
10:13:52 | gokr | Varriount: I am quite busy today, but tomorrow I can set up Linux slaves if you wish. The box is up and "waiting". |
10:27:35 | * | sanjay quit (Quit: Page closed) |
10:47:53 | * | uku joined #nimrod |
10:48:43 | * | dapz quit (Quit: Textual IRC Client: www.textualapp.com) |
10:57:48 | * | darkf joined #nimrod |
11:03:38 | * | uku quit (Ping timeout: 250 seconds) |
11:28:30 | * | darkf_ joined #nimrod |
11:32:25 | * | darkf quit (Ping timeout: 265 seconds) |
11:43:55 | * | uku joined #nimrod |
11:54:43 | * | darkf_ is now known as darkf |
11:55:38 | * | gokr_ joined #nimrod |
12:37:51 | * | def- quit (Quit: -) |
12:38:30 | * | def- joined #nimrod |
12:40:24 | * | BitPuffin joined #nimrod |
12:50:48 | * | q66 joined #nimrod |
13:05:01 | * | flaviu joined #nimrod |
13:15:40 | * | gokr_ quit (Remote host closed the connection) |
13:15:55 | * | gokr_ joined #nimrod |
13:54:54 | * | foodoo joined #nimrod |
13:54:56 | Varriount | gokr: Ok, thanks. |
13:55:42 | * | moritz left #nimrod (#nimrod) |
13:57:59 | Varriount | dom96, Araq: A webhook needs to be added to the Nimrod Github repo to point to http://178.62.143.63:9001 |
13:59:50 | foodoo | hm. Is it still "Nimrod" or will it be called "Nim" in the future? |
14:00:34 | Varriount | foodoo: It's Nim, but it's going to be a while until I actually remember to call it that. |
14:01:09 | Varriount | foodoo: For one thing, all my local... 'Nim' directories are named 'Nimrod' |
14:07:19 | dom96 | Varriount: I don't have access to that unfortunately. |
14:08:22 | dom96 | Araq: Should I rename nimrod-code to nim-code? |
14:11:20 | dom96 | -+ |
14:11:26 | dom96 | + |
14:11:46 | dom96 | oops |
14:12:02 | foodoo | You ascii art brush dropped into the IRC window |
14:23:40 | * | BlaXpirit-UA joined #nimrod |
14:24:52 | * | BlaXpirit quit (Ping timeout: 245 seconds) |
14:29:22 | * | foodoo quit (Quit: leaving) |
14:29:33 | rpag | the diff ascii brush |
14:36:19 | * | rpag quit (Ping timeout: 250 seconds) |
14:42:36 | * | darkf quit (Remote host closed the connection) |
14:49:27 | * | rpag joined #nimrod |
15:11:17 | * | BitPuffin quit (Ping timeout: 264 seconds) |
15:11:52 | * | BitPuffin joined #nimrod |
15:14:40 | * | kemet joined #nimrod |
15:45:38 | * | johnsoft quit (Ping timeout: 255 seconds) |
15:47:50 | * | johnsoft joined #nimrod |
16:00:23 | * | untitaker quit (Ping timeout: 250 seconds) |
16:06:31 | * | untitaker joined #nimrod |
16:07:47 | * | johnsoft quit (Ping timeout: 245 seconds) |
16:08:40 | * | johnsoft joined #nimrod |
16:24:25 | * | johnsoft quit (Ping timeout: 244 seconds) |
16:24:36 | * | johnsoft joined #nimrod |
16:25:47 | * | irrequietus joined #nimrod |
17:22:39 | * | xenagi joined #nimrod |
17:22:57 | * | xenagi left #nimrod (#nimrod) |
17:40:06 | * | perturbation joined #nimrod |
17:40:46 | perturbation | I forgot who it was that I was talking to on here about this... but I have to apologize about Fortran |
17:41:03 | perturbation | Turns out Fortran '03 (well, '95 onwards) is so much nicer than Fortran '77 |
17:41:56 | perturbation | I mean, I'm still not going to use it for day-to-day stuff, but it's still much cleaner than I thought it was |
17:43:42 | * | whoot joined #nimrod |
17:44:42 | whoot | How can I "gc-safely" iterate over a seq? |
17:45:36 | dom96 | whoot: Are you looking to iterate over it in multiple threads? |
17:46:17 | whoot | yes |
17:46:29 | whoot | I'm trying to use that from a handler of an async http server |
17:46:39 | whoot | but I reduced my problem to http://pastebin.com/dfWYAVez |
17:47:04 | dom96 | Async does not use threads implicitly. |
17:47:21 | dom96 | The reason that is not gc safe is because `s` is a global var. |
17:48:00 | dom96 | It should work as long as you don't compile with --threads:on |
17:48:14 | dom96 | if you want to use threads then declare s as a {.threadvar.} |
17:48:20 | dom96 | http://build.nimrod-lang.org/docs/manual.html#threadvar-pragma |
17:48:22 | whoot | yes, I was trying to spawn a thread per request. maybe it's absolute overkill |
17:48:52 | whoot | my bad, but I do want to be reading a global variable |
17:49:02 | dom96 | If you're using async await at the same time then that sounds like a bad idea. |
17:49:53 | dom96 | Async await may blow up completely with spawn. |
17:50:08 | dom96 | If it doesn't then you're very lucky. |
17:52:01 | whoot | ok, disregard the whole threading stuff I mentioned |
17:52:34 | * | kniteli joined #nimrod |
17:52:55 | whoot | same problem happens if I try to iterate over a global seq from a request handler |
17:53:46 | dom96 | are you compiling with --threads:on? |
17:53:52 | whoot | yes, is that bad? |
17:54:00 | * | BitPuffin quit (Ping timeout: 250 seconds) |
17:54:12 | dom96 | It's not bad. But it's the cause of this error. |
17:54:32 | whoot | I'd like ideally to get the http server to use all processors, that's why I passed --threads:on |
17:54:41 | * | q66[lap] quit (Read error: Connection reset by peer) |
17:54:44 | whoot | but then I can't figure out how to iterate over this seq |
17:54:46 | dom96 | Currently that's not supported. |
17:55:23 | * | q66[lap] joined #nimrod |
17:58:32 | * | kniteli quit (Ping timeout: 265 seconds) |
18:00:56 | whoot | Thanks for your help dom96. So if i understand correctly, it's unsupported because the async http server uses await and that does not play nice with spawn as of today. Did i understand correctly? |
18:02:06 | dom96 | Yeah. Getting this to work is on our roadmap so we will hopefully have this functionality in the future. |
18:02:33 | whoot | ok, thanks once again |
18:02:38 | dom96 | Async await is still in its infancy. |
18:02:58 | whoot | so to have a multithreaded http server i shouldnt use the async one |
18:03:17 | whoot | nvm I'll just figure it out myself |
18:10:01 | * | kniteli joined #nimrod |
18:21:48 | * | kniteli quit (Ping timeout: 265 seconds) |
18:27:07 | * | Jesin quit (Quit: Leaving) |
18:29:30 | NimBot | nimrod-code/nimforum new_async 114d322 Dominik Picheta [+0 ±4 -0]: Redesigned front page.... 4 more lines |
18:32:28 | * | wayne left #nimrod (#nimrod) |
18:32:56 | * | kniteli joined #nimrod |
18:43:56 | flaviu | whoot: If you don't care about automatic memory management, you can do createShared(T, size) |
18:44:20 | flaviu | http://nimrod-lang.org/system.html#createShared,typedesc |
18:49:41 | * | kemet quit (Quit: Instantbird 1.5 -- http://www.instantbird.com) |
18:49:54 | * | kniteli quit (Ping timeout: 272 seconds) |
18:50:47 | * | perturbation quit (Quit: Leaving) |
18:50:48 | * | kemet joined #nimrod |
18:54:07 | * | Jesin joined #nimrod |
19:00:11 | Araq | dom96, dom96_ name it "nim-lang" then, makes more sense, right? |
19:00:34 | dom96 | ok. But you'll need to fix the readme. |
19:00:39 | Araq | also I don't know why you keep saying that async doesn't work with spawn |
19:01:01 | Araq | it should work |
19:01:31 | dom96 | I never said it doesn't work |
19:01:35 | dom96 | I said it's not supported. |
19:02:01 | dom96 | I've never tested it so I don't know. |
19:02:10 | Araq | ah ok |
19:02:15 | * | kniteli joined #nimrod |
19:02:24 | Araq | well we should test it |
19:03:09 | dom96 | I renamed it. |
19:04:37 | Araq | ok, so what do I need to do? |
19:06:27 | * | BitPuffin joined #nimrod |
19:07:40 | gokr_ | whoot: Check my article about spawningserver |
19:08:38 | gokr_ | spawn uses a threadpool btw |
19:10:12 | Varriount | gokr_: The build slave shouldn't need any special requirements; The only things that are required are the buildbot software, and git. |
19:10:51 | Varriount | You'll need a password and an address to connect to - I'll PM you the information. |
19:11:00 | gokr_ | sure |
19:11:02 | * | kniteli quit (Ping timeout: 265 seconds) |
19:17:10 | dom96 | Araq: change all http://github.com/nim-code/* urls to http://github.com/nim-lang/* |
19:17:23 | Araq | where? |
19:17:36 | * | saml_ joined #nimrod |
19:18:13 | * | lbmn joined #nimrod |
19:18:33 | Araq | hi lbmn welcome |
19:18:37 | Araq | dom96: http://forum.nimrod-lang.org/t/622 |
19:19:50 | gokr_ | nice!! |
19:20:07 | dom96 | well, I was just going to redirect to google. |
19:22:20 | Varriount | Araq: Have you added the web hook for the repository? |
19:22:29 | * | kniteli joined #nimrod |
19:24:29 | flaviu | Varriount: You can actually even get by withou git |
19:25:06 | * | tillzy joined #nimrod |
19:25:09 | Varriount | flaviu: Not with the buildbots. The master configuration uses Git. |
19:26:15 | Araq | Varriount: "just the push event" or "everything" or "individual events"? |
19:26:24 | Varriount | Just the push event. |
19:26:57 | Araq | "Okay, that hook was successfully created. We sent a ping payload to test it out! Read more about it at https://developer.github.com/webhooks/#ping-event." |
19:27:40 | whoot | thanks gokr_ |
19:28:01 | * | saml_ quit (Ping timeout: 265 seconds) |
19:28:09 | Varriount | Araq: This ping will fail - the bot will only respond to push events. |
19:30:16 | Araq | Varriount: ok, well, that shouldn't matter. |
19:32:07 | lbmn | Is there an alternative json library binding for nimrod? |
19:32:33 | lbmn | (Did you catch the decoding benchmark results I've mentioned last time?) |
19:35:09 | Varriount | lbmn: Not really. I mean, if there's a fast C or C++ json library out there, it can be wrapped, but I don't know of any currently existing alternative wrapper. |
19:35:45 | Varriount | Aside from that, I would really rather have the pure-nimrod implementation upgraded. |
19:37:00 | Varriount | Araq: On a related note, I wasn't able to implement the string assignment optimization - the compiler kept crashing on bootstrap. |
19:40:48 | lbmn | With all optimizations turned on, Nimrod's default json decoding speed is behind many scripting languages, which for some cases would mean the whole program would be slower. |
19:41:10 | lbmn | I realize that some json decoders do more than others, but still... |
19:41:46 | * | saml_ joined #nimrod |
19:41:46 | flaviu | lbmn: Can you post benchmark programs? |
19:42:01 | lbmn | Too ugly for github right now... |
19:42:19 | flaviu | private gist? |
19:42:27 | Varriount | Does the embedded profiler still work? |
19:42:55 | flaviu | linux perf is really excellent, I'll be testing it with that |
19:43:18 | flaviu | You can't beat instruction-level data |
19:43:55 | Varriount | Araq: Now that the main build configuration is done, I'll be working on the manual builds, and then the web interface. |
19:44:05 | lbmn | Just giving a bit of feedback. Switching json libraries would be a low-effort way of getting a noticeable performance boost. |
19:46:15 | lbmn | The program is as simple as it gets - http://pastebin.com/nBg05sam |
19:47:15 | * | q66[lap] quit (Read error: Connection reset by peer) |
19:48:50 | * | saml_ quit (Ping timeout: 250 seconds) |
19:49:31 | * | q66[lap] joined #nimrod |
19:50:06 | lbmn | luajit version that runs in less than half the time (1.26s, vs nimrod's 2.95s, or 12.50s without optimizations) - http://pastebin.com/CUfxauQx |
19:51:09 | Varriount | lbmn: Yes, switching libraries would be a low effort way, but then users would be required to compile the luajit library. |
19:51:41 | flaviu | Varriount: There are other libraries, including some specificly designed for embedding |
19:51:50 | flaviu | Some are likely faster than luajit |
19:52:42 | Varriount | Hrmf. I'd still rather port a library over. There should be some things we provide that shouldn't rely on 3rd party libraries. |
19:53:17 | lbmn | I mean including nimrod bindings for a fast C json library like jsmn, ujson, yajl, etc. |
19:53:38 | flaviu | Varriount: even porting one isn't going to be too good for javascript. |
19:53:55 | flaviu | When you should the browser's C version |
19:55:00 | * | kniteli quit (Ping timeout: 244 seconds) |
19:55:01 | * | kemet quit (Quit: Instantbird 1.5 -- http://www.instantbird.com) |
19:56:39 | lbmn | (Node.JS runs that benchmark in 1.90 with require, 1.93 with JSON.parse.) |
20:00:35 | lbmn | Strange enough, all "C-alternative" compiled languages lag behind, not just nimrod: rust 3.2s, go 3.63. |
20:01:53 | * | saml_ joined #nimrod |
20:03:13 | * | askatasuna joined #nimrod |
20:04:16 | gokr | flaviu: The cjson library that the luajit test uses is actually a pure C library. |
20:04:23 | gokr | Its the one I pointed out yesterday. |
20:05:15 | gokr | Since LuaJIT calls into C just as fast as C does, result is ... fast. |
20:06:39 | * | kniteli joined #nimrod |
20:06:53 | * | kemet joined #nimrod |
20:16:37 | Araq | gokr: iirc luajit calls into C faster as C does because it can inline even more |
20:16:45 | Araq | *faster than |
20:16:52 | gokr | Even that yeah. |
20:17:14 | flaviu | is v8 somehow cheating? |
20:17:35 | flaviu | It spends all the time fetching the file, parsing the json is instant |
20:18:06 | Varriount | lbmn: At least nimrod is faster than rust. |
20:19:40 | gokr | I peeked at the source of ultrajson on github, hehe, lots of memcpy etc. Looks heavily optimized. |
20:20:09 | gokr | https://github.com/esnme/ultrajson/tree/master/lib |
20:20:12 | Varriount | gokr: Was that sarcasm? |
20:20:43 | flaviu | Varriount: That sort of argument is pretty terrible. "This isn't the slowest, so use it" |
20:22:09 | Araq | flaviu: that's a misrepresentation of Varriount's argument. |
20:22:34 | Araq | so where can I find this json test file? |
20:22:45 | Araq | but I think I know why it's slow in Nim anyway |
20:22:54 | flaviu | Araq: I don't know about his, but I'm using https://github.com/zemirco/sf-city-lots-json/blob/master/citylots.json |
20:23:12 | gokr | Varriount: No... and I kinda misread the code anyway. But its meant to be fast anyway. |
20:23:12 | flaviu | 190MB |
20:23:49 | Araq | ok, just as I suspected |
20:24:10 | Araq | lol I don't even need profilers anymore :P |
20:24:45 | * | Jehan_ joined #nimrod |
20:25:22 | Araq | hi Jehan_. I really like my implementation better. but can you give some hints as to why everything in it is wrong? |
20:26:00 | * | kniteli quit (Ping timeout: 244 seconds) |
20:26:16 | Jehan_ | Araq: Umm, I'm not saying that "everything is wrong". |
20:26:45 | Araq | well when I read your commit message, that surely reads like "bug here and here and here ..." |
20:26:50 | Jehan_ | Except for the attempt to load balance at the userland level, which I think is just a waste of CPU cycles. |
20:27:11 | Jehan_ | Yeah, race conditions, such as the one that got you stuck in a deadlock. |
20:27:24 | Araq | well but that's exactly the feature that gave gokr good results |
20:28:08 | Araq | it's also similar to what .NET's thread pool does |
20:28:43 | lbmn | One C JSON libraries benchmark - http://lionet.livejournal.com/118853.html (In Russian, scroll down to results.) |
20:29:49 | Jehan_ | The biggest problem I see with the code is that it uses a mix of techniques that are really, really difficult to validate as being correct. |
20:30:29 | * | tillzy quit (Quit: tillzy) |
20:30:41 | Jehan_ | The barrier code, for example, is still not correct, as far as I can tell. |
20:31:06 | Jehan_ | It happens to not break on x86 processors, but I wouldn't bet on it for other types of processors. |
20:31:34 | lbmn | Rust version in that benchmark is incomplete, I gave up before I figured out how to print the count (which shouldn't affect it by much). Compiled with `rustc --opt-level 3`. http://pastebin.com/pK4KFQEv |
20:34:09 | lbmn | I'm just saying, people do a cost-benefit analysis in their head before switching from their ScriptLang to Nimrod. There has to be a performance boost, and usually there is, but with a JSON-heavy program there won't be. |
20:34:55 | Jehan_ | Then there's the problem with nimSpawn. It's not that I actually think it has a bug (at least not on TSO processors), there's just no way I can validate its correctness, and hence I don't trust it, because you can't reliably test parallel code (assuming it isn't specifically designed for being testable). |
20:35:00 | * | tillzy joined #nimrod |
20:35:14 | flaviu | lbmn: You're using an old version of rust |
20:36:36 | Jehan_ | People stick with traditional monitor semantics for most stuff because the semantics are extremely well understood and can be reasoned about. Doing dirty reads and using lower-level primitives has to be done with an extreme level of care if you want to be able to trust the code. |
20:36:56 | Araq | Jehan_: ok, please tell me *why* the barrier is still not correct |
20:37:11 | lbmn | The nightly one was FUBAR. |
20:37:29 | * | kniteli joined #nimrod |
20:37:34 | Araq | lbmn: there is no argument here. one way or the other we'll improve json parsing. |
20:37:45 | gokr | Araq: Curious what you "knew" was the problem in the JSON lib :) |
20:37:55 | Jehan_ | Araq: Because on some architectures, barrierLeave may not see the most recent value of b.entered |
20:38:42 | flaviu | lbmn: .12 works though |
20:39:01 | Jehan_ | To be precise, it's again an "I can't be sure" thing, because it may still accidentally turn out to be correct. But I cannot trust the code. |
20:39:13 | Araq | Jehan_: there is a comment in barrierEnter that explains why I think there is no "fence" necessary |
20:39:14 | * | Jehan_ quit (Quit: Leaving) |
20:39:36 | * | Jehan_ joined #nimrod |
20:39:53 | lbmn | I don't like Rust anyway. Nimrod has the best syntax of any language on the system's half of Ousterhout's tennis court... |
20:40:11 | Jehan_ | Araq: I know. The problem is that ordering memory accesses is not enough. |
20:40:24 | Jehan_ | Just because you have a fence somewhere doesn't mean stuff gets written to memory. |
20:40:31 | Jehan_ | It means that reads/write don't get reordered. |
20:40:35 | Araq | er ... well |
20:40:43 | Araq | that's what I thought it does |
20:41:24 | Jehan_ | Fences basically mean that reads/writes before the fence don't get swapped around with reads/writes after. |
20:41:36 | Araq | I see |
20:41:40 | Jehan_ | There's no generic "flush all stuff to memory" instruction on all processors. |
20:41:45 | Jehan_ | Now, here's the thing. |
20:42:16 | gokr | Btw, I don't want to bother Araq with questions on the exception system in Nim - anyone care to comment on it - say contrasted with the Java model? |
20:42:41 | Jehan_ | It may still work because stuff like thread creation creates its own barriers. |
20:43:00 | Jehan_ | And so forces events to be sequentialized at a global level so that it still works. |
20:43:58 | Araq | well yes, but on the other hand, I tested it and actually *debugged* it. |
20:44:09 | Araq | so it certainly didn't always work |
20:44:25 | Jehan_ | Araq: Yeah. The thing is, you tested it on an Intel or AMD processor. |
20:44:43 | Jehan_ | They're TSO processors, which is nearly as good as "always sequentially consistent". |
20:45:02 | wan | lbmn: jsmn looks awesome. On its page: "no dynamic memory allocation". They must be spliting the original string, no wonder it's the fastest. No copy! No alloc! I'm not even sure Nim can do that (certainly not with a json module backed by a hash table using ref). |
20:45:09 | Jehan_ | The downside is that a full fence instruction on such a processor is really, really expensive. |
20:45:34 | Jehan_ | Pipeline stall while waiting for data written to memory in the worst case. |
20:46:36 | Jehan_ | On the other hand, pure read barriers and pure write barriers are nops. |
20:46:51 | Jehan_ | And overall it's much saner to deal with than, say, ARM. :) |
20:49:22 | Jehan_ | In the general case, however, if you want to make sure that a write in one thread becomes visible to another thread, then the write must be succeeded by a handshake operation, and the handshake operation in the other thread must precede the read that that thread does. |
20:49:34 | Jehan_ | This is what condition variables and semaphores accomplish. |
20:50:15 | Araq | wan: Nim can easily do it. now ok, the stdlib doesn't support it, but then C barely even has a stdlib. so you end up with C-like Nim code. Which still has advantages over C. |
20:52:34 | Araq | Jehan_: I'm not even sure I like semaphores better than CAS+races. The thing is ... with a semaphore I have to make liveness guarantees all the time. and I dunno how you can reason about *that*. |
20:52:43 | gokr | Good page on jsmn: http://zserge.com/jsmn.html |
20:52:59 | gokr | So it simply points to where the data starts/ends in the original string. |
20:53:24 | Jehan_ | What liveness guarantees do you mean? |
20:53:58 | Araq | whenever I call 'await' there is no timing involved. I need to ensure 'signal' really will be called eventually. |
20:54:33 | Jehan_ | That's honestly why locks + condition variables are often the better solution. |
20:54:47 | lbmn | It would be awesome to have many alternatives for performance-critical libraries like json, bonus points if they share the same method names / etc so switching between them is easy. |
20:55:38 | lbmn | Like python's anyjson (although pypy json beats CPython's best C libraries for some reason). |
20:55:58 | wan | Araq: I'm probably unexperienced on that side of Nim, I had troubles before. I can get a pointer with addr(a), but doing something with it, especially like casting a ptr to an array and using it... I must try again. |
20:56:11 | Araq | Jehan_: well yeah but I came up with the CondVar because locks+condition variables have their own problems, at least for me. |
20:56:14 | lbmn | Then people can switch based on circumstances, like file size, memory limits, number of CPU cores, etc. |
20:56:26 | Jehan_ | lock; while not predicate: wait; unlock matches with lock; update predicate; signal; unlock is a pretty universal mechanism how two threads can agree on pretty much arbitrarily complex predicates while maintaining thread safety. |
20:56:27 | * | Sembei joined #nimrod |
20:57:07 | flaviu | wan: You can't cast a pointer to an array, a pointer is referencing a location memory, while an array is directly memory |
20:57:34 | flaviu | Arrays are c-style, not java style |
20:57:54 | Jehan_ | Araq: That's essentially a semaphore, and while semaphores have their uses, they also provide somewhat weaker guarantees. |
20:58:16 | * | Pisuke quit (Ping timeout: 258 seconds) |
20:58:41 | * | johnsoft quit (Ping timeout: 260 seconds) |
20:58:52 | Araq | wan: well I can do it and I guess we need better docs, as usual |
20:58:56 | * | johnsoft joined #nimrod |
20:59:11 | wan | flaviu: so you can't do `somevar[count] = ...` if somevar is a ptr? That's what I was trying to do at the time, if I recall correctly. |
20:59:40 | flaviu | a ptr array[T, ...]? |
20:59:41 | Araq | wan: use a ptr to an array to do that. no big deal at all. |
20:59:58 | wan | As I said, I must try again. |
21:00:04 | Jehan_ | The biggest limitations are that (1) you have no atomicity guarantees for the state you're operating on and (2) the number of signals must match the number of waits (which can lead to additional bookkeeping). |
21:00:57 | gokr | wan: jsmn is on the other hand only a tokenizer. Doesn't do anything more. |
21:06:06 | Araq | Jehan_: wie auch immer. I don't want to merge your PR because it feels like losing all my work and I don't want to dismiss your PR because that would lose your work... Can't we have some -d:paranoid flag for weak CPU architectures? |
21:07:04 | Jehan_ | Araq: First, don't worry about my work. That was just a couple of hours of coding, and it was faster than writing it up as text. |
21:07:13 | wan | gokr: what else should a json parser do? |
21:07:37 | gokr | I mean, it doesn't do number parsing for example. And not char escape codes etc. |
21:07:57 | gokr | IIUIC |
21:09:15 | wan | I've checked how jsmn "no alloc" works: you need to give it space for tokens yourself (either on the stack or heap, beforehand). So, no arbitrary number of nodes, but it's still cool |
21:09:23 | * | johnsoft quit (Ping timeout: 258 seconds) |
21:09:41 | Jehan_ | But, the whole current approach worries me in that it seems to be too much concerned with microoptimization (without actual evidence that there's a payoff) rather than putting correctness first. |
21:10:18 | * | johnsoft joined #nimrod |
21:10:20 | wan | gokr: there is a test with escape stuff, I think it covers it |
21:11:06 | wan | but yeah, number parsing is important |
21:11:56 | gokr | I just read some discussion on it. |
21:13:00 | Araq | Jehan_: well sure. on the other hand the barrier does pay off according to you. and avoiding allocShared has its advantages too. |
21:14:13 | Araq | I mean, currently spawn itself does no allocation whatsoever (I think) and so ensures bounded memory usage. which some domains really require. |
21:15:36 | Jehan_ | Umm, if there's a performance difference, it's in the low single digit percentage range, from what I've been able to tell (may be different on other OSs, I don't know). |
21:16:07 | Jehan_ | To improve performance, I'd mostly avoid recreating the semaphore each time. |
21:16:47 | Jehan_ | You could cache a thread-local instance and only create a new one in the rare case that you have nested parallel statements. |
21:17:20 | Jehan_ | The biggest problem there IMO is still the necessary synchronization in nimArgsPassingDone and the serialization bottleneck that entails, though. |
21:19:43 | Araq | ok, but yours is also not exception safe and so would require an implicit try-finally which we don't generate yet, adding to its cost |
21:21:57 | Araq | apart from that I don't really use micro optimizations for the sake of it. I find the 'cas' in the code much easier to reason about than what I had before. |
21:22:32 | Jehan_ | What precisely about it is not exception safe that isn't already a problem in the existing code? |
21:23:27 | Araq | the cond var stuff that requires freeing only happens in 'closeBarrier', it's not spread out. |
21:24:13 | Araq | so we can get away without any 'try-finally', but even if we wouldn't, it would only happen if b.left != b.entered |
21:24:22 | Jehan_ | I see. |
21:25:18 | Jehan_ | That can be fixed, though. |
21:25:50 | * | kemet quit (Quit: Instantbird 1.5 -- http://www.instantbird.com) |
21:26:12 | Jehan_ | The primary problem is that the check for b.left == b.entered is not safe. |
21:30:17 | wan | lbmn: for me, using -d:release, with the 190MB file, nim takes 4.9s and luajit takes 4.2s. I think it's pretty good. Have you tried forgetting all your switches and just using -d:release? |
21:34:03 | wan | It does take a lot more memory though, I think luajit is around 600-700MB, and nim goes to 1150MB for that same file of 181MB |
21:37:29 | * | kniteli quit (Ping timeout: 265 seconds) |
21:38:21 | Jehan_ | Araq: Somewhat unrelated, is there an atThreadExit/atThreadInit functionality? |
21:39:10 | Jehan_ | I couldn't find any, but that obviously doesn't mean it's not there. |
21:41:32 | gokr | wan: Then perhaps we can simply tune what we have for now. |
21:41:48 | * | Trustable joined #nimrod |
21:42:54 | * | whoot quit (Quit: Leaving) |
21:46:37 | * | irrequietus quit () |
21:48:38 | Jehan_ | Anyhow, have to get up early tomorrow, good night. :) |
21:48:39 | * | Jehan_ quit (Quit: Leaving) |
21:48:54 | * | kniteli joined #nimrod |
21:55:51 | lbmn | My Facebook Page cover image now mentions Nimrod... https://www.facebook.com/copyfree.org (To be updated soon...) |
21:56:20 | lbmn | Gotta disconnect. If you need me, msg me on Facebook or [email protected]... |
21:56:41 | * | lbmn quit (Quit: Leaving) |
21:57:03 | gokr | Anyone here using gitlab btw? |
22:06:00 | * | kniteli quit (Ping timeout: 265 seconds) |
22:08:25 | gokr | Varriount: Can you give me just an itsy bitsy more info on what I should do to set up a slave for you? :) |
22:11:53 | Varriount | gokr: Well, you need to install the buildot slave package ('pip install buildbot slave'), then create a slave ('buildslave create-slave <slave_dir> 178.62.143.63:9989 linux-x32-slave-1 <password>') |
22:12:25 | Varriount | then start the slave, ('buildslave start <slave_dir>') |
22:12:30 | flaviu | Has anyone managed to get nim to parse the file? |
22:12:44 | Varriount | flaviu: Parse what file? |
22:12:50 | flaviu | the json |
22:12:56 | gokr | Ok, and... this is a 64 bit Linux, but I presume the build will use the -m32 flags etc? |
22:13:30 | Varriount | gokr: The two linux build slaves are named 'linux-x32-slave-1' and 'linux-x64-slave-1' |
22:14:07 | gokr | Which version are you on? 0.8.9? |
22:14:33 | Varriount | Yep. |
22:15:35 | Varriount | gokr: Other than that, you'll need git for checking out the repositories, and sqlite for the testers. |
22:15:47 | gokr | yeah |
22:16:14 | gokr | Wonder if I should do this in docker or not. Will try first "regularly". |
22:17:02 | Varriount | After that, the slaves should be relatively low-maintenance. All the build steps are given by the master, and any auxilary scripts used locally are downloaded via git. |
22:17:21 | * | kniteli joined #nimrod |
22:17:47 | Varriount | gokr: If you can, have the build slaves restart automatically if they shutdown due to being disconnected from the build master. |
22:18:15 | gokr | ok |
22:19:45 | Varriount | gokr: Thanks for the support, it's greatly appreciated. |
22:20:11 | * | Matthias247 quit (Read error: Connection reset by peer) |
22:20:37 | gokr | We are going to use Nim, so we should "chip in". |
22:20:58 | wan | flaviu: I tried the nim program that lbmn did, run fine with the big file |
22:21:11 | Varriount | gokr: You can see builder status on http://178.62.143.63:8010/waterfall |
22:21:20 | flaviu | I'm getting a syntax error on line 1 |
22:21:27 | gokr | Varriount: right |
22:25:14 | wan | flaviu: did you paste it from the RAW pastebin area? The first line is fine, it's just import json, strutils |
22:25:41 | flaviu | No, a syntax error in the json file. Really weird, maybe I should pull the latest nim |
22:26:48 | wan | I wgetted the file from the raw github link. Maybe yours stopped half-time while downloading? |
22:26:51 | * | kniteli quit (Ping timeout: 265 seconds) |
22:28:32 | flaviu | I did exactly the same |
22:29:00 | flaviu | Yep, the file looks good to me. |
22:29:26 | wan | 158346af5a90253d8b4390bd671eb5c5 citylots.json |
22:29:33 | wan | (md5sum) |
22:30:32 | flaviu | It's the same file, although you should migrate to sha256 asap |
22:34:17 | flaviu | input(1, 2) Error: string literal as key expected expected |
22:34:27 | flaviu | Anyway, I'll figure it out later |
22:36:56 | wan | I'm using Nim Compiler Version 0.10.1 (2014-11-08) [Linux: amd64] if you want to know. |
22:38:19 | * | kniteli joined #nimrod |
22:41:46 | Varriount | gokr: I know why the linux builder failed - it's a bug in the master config. |
22:41:53 | gokr | ah cool |
22:41:57 | gokr | It tried at least :) |
22:42:15 | Varriount | gokr: Araq doesn't want the execute bit set for the build.sh script. :/ |
22:42:38 | gokr | Ah, yeah, I can see it in the log |
22:43:01 | gokr | I will create the 64 too. |
22:44:03 | gokr | Varriount: Same pass for 64? |
22:45:34 | gokr | So we will be building the Linux stuff in Las Vegas. :) |
22:45:36 | * | Trustable quit (Quit: Leaving) |
22:45:47 | Araq | Varriount: I don't mind the exec bit |
22:45:59 | Araq | but I think it's more robust to not rely on it |
22:46:02 | Varriount | gokr: Yes.. although I don't have experience with setting up dual builds on linux. |
22:46:19 | Araq | if it pains you go ahead and make it exec. |
22:46:21 | gokr | "dual builds"? |
22:46:30 | Varriount | gokr: 32 and 64 bit builds. |
22:46:34 | Araq | your productivity is much more important |
22:46:52 | gokr | But... the build scripts are different? |
22:46:56 | gokr | I hope :) |
22:47:02 | Varriount | Uh, not really. |
22:47:06 | gokr | Oh. |
22:47:18 | Varriount | gokr: https://github.com/nim-lang/nim-buildbot/blob/master/build_steps.py |
22:47:22 | dom96 | Why can't you simply execute `sh build.sh`? |
22:47:23 | gokr | Ok, checking |
22:47:32 | Varriount | dom96: That's what I just changed it to. |
22:49:37 | gokr | Hmmm, that log looks a bit oddish. I think I will start the slave differently. |
22:55:20 | Varriount | I'm hoping that I can modify the build steps and the build configuration enough to generate a comparitive view of the test results. |
22:55:31 | Varriount | Right now the results just print to stdout. |
22:55:44 | Araq | Varriount: the tester already supports that |
22:55:48 | Araq | I hope you know |
22:55:52 | gokr | Varriount: csources built now last time. |
22:56:17 | Varriount | Araq: The tester supports html output and json output, but only for the platform it's been tested on. |
22:56:33 | Varriount | You can't get a comparitive view of two different platforms. |
22:56:36 | gokr | Varriount: Then it failed on "nimrod c koch.nim". Should it be nimrod? |
22:57:22 | Araq | Varriount: maybe but it's easy to add support for that, the database model has the concept of a platform |
22:57:54 | gokr | Varriount: Force again, I started it differently. |
22:59:45 | gokr | And... the 64 slave failed: unauthorized login; check slave name and password |
23:00:12 | Varriount | I'll PM you the password and name again, it's possible I made a mistake. |
23:00:48 | gokr | The name for the slave seems correct. |
23:00:56 | gokr | So perhaps a different pass. |
23:01:01 | gokr | Ah, that's different :) |
23:02:23 | gokr | Now it started fine. |
23:02:48 | flaviu | Varriount: I have a arm v5 device, can you PM me the password? |
23:03:25 | Varriount | flaviu: Uh, I haven't created a slave account for that platform yet. |
23:03:50 | Varriount | flaviu: I'll pm you a set of slave credentials, just give me a bit to actually set the slave account up. |
23:04:10 | flaviu | oh, I thought it was global. That's fine |
23:05:02 | gokr | Varriount: You now... you are building "nim" with csources. |
23:05:07 | gokr | Varriount: Not "nimrod" |
23:06:15 | Varriount | gokr: On windows, the binary name generated by csources is nimrod. I guess it's different on linux |
23:06:30 | gokr | You can tell if you look at the log from the step before. |
23:06:54 | Varriount | Regardless, the normalization step should take care of that. |
23:07:47 | * | brson joined #nimrod |
23:11:28 | * | Demos joined #nimrod |
23:15:38 | * | uku quit (Ping timeout: 255 seconds) |
23:18:24 | * | uku joined #nimrod |
23:21:34 | * | tillzy quit (Quit: tillzy) |
23:22:51 | * | kniteli quit (Ping timeout: 265 seconds) |
23:25:13 | Varriount | gokr: Can you check what binaries are present in the slaves build directory? |
23:25:21 | gokr | mmm |
23:26:35 | gokr | In linux32, in bin - there is both a "nim" and a "nimrod" - both same size, but only "nim" is executable. |
23:26:47 | Varriount | Ah, that would be it.. |
23:27:08 | Varriount | gokr: Apparently python's copyfile() doesn't copy the executable bit.. |
23:27:17 | gokr | yeah |
23:28:11 | flaviu | Varriount: Sorry to bother you again, but I may have rudely killed my slave, causing problems. Can you do something about "rejecting duplicate slave" |
23:28:54 | gokr | So... is it you who are copying "nim" to "nimrod" in your build script? |
23:29:05 | Varriount | flaviu: Make sure no zombie processes are running. I can't do anything from my side other than restarting the master. |
23:29:39 | gokr | Also, docker seemingly doesn't do 32 bits on 64 yet. So for a "true" 32 bit Linux, I think we need vagrant. |
23:29:55 | flaviu | Varriount: You're right, I messed up the auto-restart functionallity |
23:33:01 | gokr | flaviu: What autorestart thingy do you use? |
23:33:08 | flaviu | gokr: systemd |
23:33:12 | gokr | ok |
23:33:27 | flaviu | I'll gist my service file once I get everything working |
23:33:58 | Varriount | flaviu: What distro are you using? |
23:34:03 | flaviu | Arch linux |
23:34:24 | * | kniteli joined #nimrod |
23:34:26 | Varriount | Ah. So Arch doesn't have a buildbot package/doesn't set up a buildmaster user for you? |
23:35:33 | flaviu | It doesn't have an official or AUR package for buildbot. I could make an AUR package, but meh, too much work. |
23:37:14 | wan | gokr: for me (on linux), csources make a nimrod executable, not nim |
23:37:28 | Varriount | flaviu: What user are you running the build slave under? |
23:37:46 | flaviu | I made a special user for it, but I need to lock it down a bit more |
23:38:31 | Varriount | wan: I'm aware of that. There's a script I run which creates a nimrod/nim executable based on whether a nim/nimrod executable already exists. |
23:41:34 | * | bjz quit (Ping timeout: 258 seconds) |
23:43:29 | * | wan quit (Quit: WeeChat 1.0.1) |
23:44:56 | Varriount | gokr: Ok, something must be off. |
23:45:12 | gokr | mm? |
23:45:51 | gokr | Ah, you changed it to nim |
23:45:56 | Varriount | Oh, wait, nevermind |
23:46:49 | gokr | Now none of them are executable :) |
23:47:02 | Varriount | Yeah, I realized that. |
23:47:06 | Varriount | >_< |
23:48:25 | flaviu | https://gist.github.com/flaviut/9ddfcb94ebe6161b3bc6 |
23:48:53 | gokr | flaviu: cool |
23:49:19 | flaviu | Seems to be working for me, but given the speed that this device moves, it'll be a while until I even finish bootstrapping |
23:49:40 | Varriount | flaviu: What device would this be? |
23:49:48 | gokr | and... exec-buildbot script? |
23:49:55 | flaviu | Pogoplug series 4 |
23:50:09 | flaviu | gokr: Just a wrapper to set up my virtualenv |
23:50:15 | gokr | ok |
23:50:19 | flaviu | Let me fix that |
23:50:23 | Varriount | gokr: *gasp*, look at the waterfall page. |
23:50:47 | gokr | :) |
23:51:39 | flaviu | Varriount: mac doesn't support x32 |
23:52:00 | flaviu | oh, wait |
23:52:06 | Varriount | flaviu: Uh, yes it does. |
23:52:28 | flaviu | All their CPUs support 64, but it looks like 32 is supported for some random reason |
23:54:25 | flaviu | tsk, tsk Varriount. You need to edit buildbot/info/* :P |
23:54:31 | flaviu | http://178.62.143.63:8010/buildslaves |
23:57:00 | Varriount | flaviu: Huh? |
23:57:03 | Araq | Varriount: does that github hook work? |
23:57:15 | flaviu | Varriount: Your Name Here <[email protected]> |
23:57:34 | Varriount | Araq: I think so. It should work... |
23:57:57 | flaviu | Varriount: And your buildbot description is "Please put a description of this build host here" |
23:58:15 | Varriount | flaviu: Details, details... |
23:58:41 | flaviu | Yes, but it looks sloppy if you don't pay attention to them |