| 00:30:23 | boydgreenfield | Can anyone point me to a good optimization/profiling checklist for Nimrod? And/or would be willing to take a look at this bitarray implementation before I push it and add it to Babel? I feel like I may be missing some low-hanging fruit, but am struggling to figure out how to further optimize something as low-level as a series of array accesses. |
| 00:33:11 | flaviu1 | Nimrod's profiler gets in the way of optimizations quite a bit, so I'd recommend -d:release --stackTrace:off |
| 00:34:51 | flaviu1 | and run through `valgrind --tool=callgrind ./yourprogram`. That way you get to see the real hotspots the integrated profiler cannot detect. |
| 00:35:06 | boydgreenfield | Ah. |
| 00:35:28 | boydgreenfield | I hadn’t even thought to use valgrind on a Nimrod program. That’s great. |
| 00:35:38 | boydgreenfield | How much overhead does stackTrace add? I haven’t been turning that off to date... |
| 00:36:38 | flaviu1 | boydgreenfield: Quite a bit: http://i.imgur.com/oN1ymIF.png |
| 00:37:56 | flaviu1 | Even more than it shows in that image, it probably thrashes the cache like crazy, and while I don't know much about compilers, it probably breaks up the pipeline quite often too. |
| 00:41:43 | boydgreenfield | Ok wow. Great, good to know. |
| 00:42:11 | boydgreenfield | What are you viewing that callgrind output in? |
| 00:42:24 | flaviu1 | KCacheGrind |
| 00:45:15 | * | DAddYE quit (Remote host closed the connection) |
| 00:45:48 | * | DAddYE joined #nimrod |
| 00:46:50 | flaviu1 | boydgreenfield: You can turn off stack traces on certain parts. `{.push stackTrace:off.} ... {.pop.}` |
| 00:47:49 | boydgreenfield | flaviu1: Great. Thanks for the help. |
| 00:50:29 | * | DAddYE quit (Ping timeout: 255 seconds) |
| 00:54:19 | * | brson quit (Ping timeout: 240 seconds) |
| 00:57:20 | boydgreenfield | flaviu1: Well with —stackTrace:off, time went from 8.6 secs —> 7.9 secs. So ~10% isn’t bad for something trivial. Out of curiosity, what flag does Nimrod use in picking its C compiler? (I’m on a Mac and want to try w/ GCC vs. Clang) |
| 00:57:57 | boydgreenfield | flaviu!: oh, only 8.3 seconds. Read the wrong line. 5%... |
| 00:58:03 | flaviu1 | boydgreenfield: Check the file that follows "Hint: used config file" |
| 00:59:52 | flaviu1 | What happens if you push lineTrace off? |
| 00:59:52 | flaviu1 | Surprising though, I wonder why valgrind reports it as 21% of the time |
| 01:00:35 | boydgreenfield | Well, this is basically all constrained by memory accesses. Which is why it’s so hard to optimize. |
| 01:00:39 | boydgreenfield | I’ll try w/o lineTrace in a sec |
| 01:01:09 | boydgreenfield | (data structure implementation somewhat akin to a Bloom filter) |
| 01:03:20 | * | bjz joined #nimrod |
| 01:03:47 | boydgreenfield | 15% slower w/ lineTrace:off? |
| 01:04:10 | boydgreenfield | (I’ll rerun, could’ve just been a quirk) |
| 01:05:33 | boydgreenfield | Ya just a quirk. Basically identical runtimes. |
| 01:06:36 | flaviu1 | I have no idea what's going on here. Can you gist the code so I can do my own tests? |
| 01:06:52 | boydgreenfield | Sure. |
| 01:07:11 | boydgreenfield | one sec. |
| 01:10:00 | boydgreenfield | Planning to put all this on Github shortly |
| 01:10:02 | boydgreenfield | Here’s the gist |
| 01:10:03 | boydgreenfield | https://gist.github.com/boydgreenfield/45c8b7b29c4ed53fa45e |
| 01:10:27 | boydgreenfield | trying to profile and speed up something like bloom_filter_w_varying_bit_demo.nim |
| 01:11:02 | boydgreenfield | Note that this is relying on MurmurHash wrapper I just pushed at https://github.com/boydgreenfield/nimrod-murmur but is not yet merged into in babel’s packages.json |
| 01:11:06 | boydgreenfield | (sorry) |
| 01:11:26 | boydgreenfield | (Hashing w/ the built-in library is like 30X slower and makes the hash operations take ~95% of the total time, making profiling hard) |
| 01:11:28 | flaviu1 | No problem |
| 01:11:46 | * | nande joined #nimrod |
| 01:12:07 | boydgreenfield | Also you don’t need to use that memfiles.nim patch — it’s only needed if you wanted the tests in bitarray.nim to pass (file permissions issue w/ mmap-backed files) |
| 01:19:47 | flaviu1 | boydgreenfield: Embarrassingly, I can't figure out how to compile murmur3.h. What line do you use? |
| 01:21:28 | * | hoverbear joined #nimrod |
| 01:25:48 | flaviu1 | nevermind, just had to do -shared |
| 01:25:54 | boydgreenfield | ah sorry |
| 01:26:42 | boydgreenfield | It didn’t compile automatically? |
| 01:26:45 | boydgreenfield | I just normally do |
| 01:26:56 | boydgreenfield | `babel install` in the directory and then it gets compiled upon import |
| 01:27:15 | flaviu1 | I dunno, I don't really use babel much |
| 01:28:10 | flaviu1 | Just curious, what CPU do you have? |
| 01:44:47 | boydgreenfield | mm |
| 01:44:49 | boydgreenfield | i’m on an i7 |
| 01:44:53 | boydgreenfield | (macbook pro retina) |
| 01:44:58 | boydgreenfield | at the moment |
| 02:03:42 | boydgreenfield | (the prior times i gave were for slightly different tests fyi) |
| 02:04:08 | * | springbok joined #nimrod |
| 02:06:39 | flaviu1 | debug: 42, 40; O3: 16.6, 16.2; no line numbers: 16.4, 16.1; no stacktrace: 11.8, 11.6; all checks off: 10.9, 10.7 |
| 02:09:38 | flaviu1 | boydgreenfield: So clang manages to optimize out the line numbers, which is good. Stack traces are ~28% of execution time, so you really want those off in important parts. Checks take ~13% off, the overhead isn't really that bad |
| 02:13:38 | boydgreenfield | flaviu1: very interesting and good to know. Now wrt the actual `[]` methods on the bitarray – that’s probably as optimal as it gets, huh? |
| 02:19:04 | flaviu1 | boydgreenfield: It isn't optimal in debug mode, but the compiler can optimize the divisions and mod without much hassle |
| 02:21:53 | boydgreenfield | flaviu1: I assumed as much, though since they’re guaranteed to be a power of 2, I guess they could be converted to shifts easily enough (just less readable) |
| 02:22:18 | flaviu1 | Yeah, don't bother optimizing that, it'll just make the code worse |
| 02:23:00 | flaviu1 | boydgreenfield: Maybe put the conditionals in `when not defined(release):`? |
| 02:24:34 | * | hoverbear quit () |
| 02:24:42 | boydgreenfield | you mean the range checks? |
| 02:25:38 | flaviu1 | Yes |
| 02:31:53 | * | flaviu1 quit (Remote host closed the connection) |
| 02:36:22 | * | DAddYE joined #nimrod |
| 03:00:38 | boydgreenfield | Interesting: that took me from 7 -> 6.5, not too bad |
| 03:01:34 | boydgreenfield | flaviu1: Is there a symbol I can use to have that only turned off when, e.g., range checks are turned off in the compiler flags? (so that I can have it be safe as a module) |
| 03:01:59 | Varriount | boydgreenfield: If you're doing benchmarks, also try changing the GC settings. If it's a short lived program, then you can compare timings with the gc off versus the gc set to markandsweep, boehm, etc. |
| 03:03:05 | boydgreenfield | flaviu1: found it. http://nimrod-lang.org/nimrodc.html#additional-compilation-switches_toc |
| 03:03:36 | boydgreenfield | (actually nevermind) |
| 03:03:43 | boydgreenfield | Varriount: Thanks. I’ll try that too. |
| 03:06:12 | Varriount | boydgreenfield: Just be ready to kill the program when running without a GC. I once tried that technique on a benchmark that did a lot of string copying, and it ate about 2 gb in 5 seconds |
| 03:07:10 | boydgreenfield | Varriount: good to know. I’m just allocating and then quitting, so shouldn’t be a problem. Do you know if there a similar symbol I can use to write a `when not defined(checks)` block to handle some manual range-checking (vs. `release`)? |
| 03:08:40 | boydgreenfield | (fyi this is what i’m optimizing: https://gist.github.com/boydgreenfield/45c8b7b29c4ed53fa45e) |
| 03:09:11 | Varriount | boydgreenfield: You want to conditionally compile part of a program, based on whether or not range checking is enabled? |
| 03:09:55 | * | hoverbear joined #nimrod |
| 03:10:32 | Varriount | boydgreenfield: I don't think there is a symbol for that. You might be able to gimmick something with compiles() though. |
| 03:12:14 | Varriount | boydgreenfield: There might also be some sort of config file thingie you can use. |
| 03:12:25 | Varriount | I'll keep looking |
| 03:13:26 | boydgreenfield | Varriout: Yes, but don’t worry about it. defined(release) is probably sufficient. |
| 03:17:03 | Varriount | Although, fixing that problem sounds like something easy enough for me to do. |
| 03:17:24 | Varriount | Hi again hoverbear. |
| 03:33:59 | hoverbear | Varriount: Hey. |
| 03:34:26 | * | Varriount just finished unit tests for file change monitoring api |
| 03:49:10 | * | Skrylar quit (Ping timeout: 258 seconds) |
| 03:55:36 | * | xenagi quit (Quit: Leaving) |
| 04:07:57 | * | Demos quit (Remote host closed the connection) |
| 04:45:50 | * | BitPuffi1 quit (Ping timeout: 255 seconds) |
| 04:50:39 | * | DAddYE quit () |
| 05:30:24 | * | BitPuffi1 joined #nimrod |
| 05:31:26 | * | BitPuffi1 is now known as BitPuffin |
| 05:34:18 | * | hoverbear quit () |
| 05:40:42 | * | Varriount puffs up BitPuffin |
| 05:41:21 | BitPuffin | get puffin |
| 06:04:29 | * | reactormonk quit (Ping timeout: 252 seconds) |
| 06:15:47 | Varriount | Status update: Nimrod's first enhancement proposal, or NEP, is up to 97 lines. (It's a style guide for the standard library) |
| 06:16:39 | boydgreenfield | Is there a way to have nimrod doc generate .rst files to then pass to, for example, readthedocs.org or similar? |
| 06:17:12 | boydgreenfield | (Just trying to figure out an easy way to quickly get nicely styled / pretty documentation) |
| 06:17:33 | Varriount | boydgreenfield: What do you mean, like, ftp them? (I have no idea how readthedocs.org does stuff) |
| 06:18:16 | boydgreenfield | Varriount: readthedocs pipeline is something like (Python or other source) -> .rst files -> HTML w/ Sphinx |
| 06:18:43 | Varriount | boydgreenfield: I think there's a switch to generate html. Lemme check |
| 06:18:46 | boydgreenfield | My sense is it’s probably easy to just use Sphinx for that 2nd step, which has some nice HTML templating options |
| 06:18:54 | boydgreenfield | Varriount: I saw a switch to generate HTML from .rst |
| 06:19:01 | boydgreenfield | but not just extract the .rst |
| 06:19:08 | boydgreenfield | (though clearly that’s being done as the intermediate step) |
| 06:19:13 | Varriount | Huh? |
| 06:19:19 | Varriount | I don't follow. |
| 06:19:52 | boydgreenfield | nimrod doc takes doc comments (rst) and just transforms them into html |
| 06:20:11 | boydgreenfield | i’m wondering if there’s a way to get it to just spit out the .rst / related index |
| 06:20:18 | boydgreenfield | maybe this is all backwards and I should just style the html |
| 06:21:34 | Varriount | boydgreenfield: Looking at nimdoc.cfg, you might be able to get away with just removing the html code from the templates. |
| 06:21:46 | boydgreenfield | Will try that. Thx. |
| 06:22:16 | Varriount | I don't know if that will work. I could swear there was an rst thingie |
| 06:22:38 | Varriount | I mean, it knows how to generate Tex formatting |
| 06:28:55 | Varriount | boydgreenfield: Are you sure that the doc generator doesn't already output rst files? |
| 06:30:13 | boydgreenfield | Varriount: I mean, not into my working directory at least. I assume it *does* generate the rst — it just may be ephemeral. |
| 06:30:48 | boydgreenfield | I’ll dig into it tomorrow. I’d just like a way to mix the automatically documented API-ish docs w/ something a bit richer. |
| 06:31:08 | boydgreenfield | (Or more precisely, more descriptive / not fully appropriate to be in docstrings) |
| 06:35:18 | Varriount | boydgreenfield: Maybe, if it doesn't exist, you could add the option to output RST to the docgen (and possibly fix doc2) |
| 07:54:50 | milosn | morning |
| 07:55:18 | * | milosn scrolls up |
| 07:55:57 | milosn | umm, sorry to repeat myself, where can i find fork() proc? i dont think anyone asnwered yesterday |
| 07:56:00 | milosn | :) |
| 07:56:20 | * | milosn would expect to find it in system |
| 07:56:24 | milosn | or os |
| 08:04:23 | * | zahary quit (Quit: Leaving.) |
| 08:05:44 | * | nande quit (Read error: Connection reset by peer) |
| 08:15:39 | * | io2 joined #nimrod |
| 09:42:34 | * | Skrylar joined #nimrod |
| 09:42:42 | Skrylar | well that teaches me not to do extensive research |
| 09:43:08 | Skrylar | mirc complained so i went "eh why not, it doesn't mention that it activates so whats the harm" |
| 09:43:33 | Skrylar | ... So apparently it does, they just make sure to not mention it anywhere important like the buy page or the top half of the EULA or the registration window |
| 09:57:14 | * | kunev joined #nimrod |
| 10:47:35 | Araq | milosn: I think you can find fork in the posix module, but don't use it, it's crap |
| 10:51:57 | Araq | for instance, fork() is not compatible with Cocoa (Mac OS X) |
| 10:56:50 | milosn | Araq: for what i need it, it needs to work on linux |
| 10:57:02 | * | milosn not bothered about OSX |
| 10:57:17 | milosn | thanks in any case, ill look at posix |
| 10:58:31 | Araq | use osproc instead |
| 10:58:46 | Araq | it's usually more efficient than fork too |
| 11:09:28 | milosn | ill have a look |
| 12:28:03 | Skrylar | to be fair, what GUI is compatible with fork? |
| 12:28:26 | Skrylar | you're basically getting a shadow copy of the program state when you fork, which puts you in a weird position |
| 12:28:59 | Skrylar | coupled with most GUIs being single-threaded scene graphs.. |
| 12:31:36 | * | darkf quit (Quit: Leaving) |
| 12:39:08 | EXetoC | no/partial sensitivity saves a lot of time when creating bindings |
| 12:46:32 | EXetoC | it's mostly just foo_bar_t that's difficult to deal with, but a regex can at least turn that into Tfoo_bar |
| 12:51:53 | EXetoC | "Multi-line procedure declarations/argument lists should continue on the same column as the preceding brace" do you guys prefer that to, say, 2-3 indents to distinguish it from the body? |
| 13:47:22 | * | askatasuna joined #nimrod |
| 14:11:45 | EXetoC | no one has time for bikeshedding? |
| 14:17:47 | * | hoverbear joined #nimrod |
| 14:24:41 | * | hoverbear quit () |
| 14:36:43 | * | BitPuffin quit (Ping timeout: 240 seconds) |
| 14:48:55 | * | hoverbear joined #nimrod |
| 14:54:52 | * | hoverbear quit () |
| 15:11:25 | * | bjz_ joined #nimrod |
| 15:12:17 | * | bjz quit (Ping timeout: 250 seconds) |
| 15:16:50 | * | nande joined #nimrod |
| 15:17:10 | * | bjz_ quit (Ping timeout: 276 seconds) |
| 15:45:24 | EXetoC | unittest.check # comment -> ident !"check" |
| 15:47:21 | EXetoC | will this remain true? the macro does this "let checked = callsite()[1]". I don't know if zahary expected it to work at all times, or if he just didn't consider comments |
| 15:49:25 | EXetoC | "block: # todo" will be disallowed at some point, right? in which case this issue should disappear |
| 15:55:40 | * | BitPuffin joined #nimrod |
| 15:55:59 | * | Varriount|Mobile joined #nimrod |
| 16:01:23 | Varriount|Mobile | EXetoC: It would be more appropriate to say that, at some point, the comment will not be included in the AST |
| 16:03:47 | EXetoC | you're right |
| 16:04:05 | * | untitaker quit (Ping timeout: 252 seconds) |
| 16:05:29 | fowl | EXetoC, if you are doing check something: # just use check(somethign) |
| 16:07:35 | * | io2 quit (Quit: ...take irc away, what are you? genius, billionaire, playboy, philanthropist) |
| 16:08:07 | * | io2 joined #nimrod |
| 16:09:31 | * | untitaker joined #nimrod |
| 16:11:47 | EXetoC | it's just a minor bug in that macro because of the current comment rules |
| 16:14:27 | * | kunev quit (Quit: leaving) |
| 16:14:50 | * | hoverbear joined #nimrod |
| 16:34:07 | dom96 | hallo |
| 16:34:35 | EXetoC | hi |
| 16:51:17 | * | boydgreenfield quit (Quit: boydgreenfield) |
| 16:56:09 | * | BitPuffin quit (Ping timeout: 258 seconds) |
| 16:58:32 | * | hoverbea_ joined #nimrod |
| 17:00:42 | milosn | if starting new networking project that needs ssl, which module is best at the moment? |
| 17:01:39 | * | hoverbear quit (Ping timeout: 252 seconds) |
| 17:03:20 | milosn | net? |
| 17:04:10 | milosn | sockets? |
| 17:04:19 | milosn | any input is welcome :) |
| 17:05:49 | * | DAddYE joined #nimrod |
| 17:05:52 | fowl | http client or server? both are available in the std lib |
| 17:06:40 | milosn | it not http, its EPP server ... length prefixed XML-ish protocol we use for provisioning domain names |
| 17:07:10 | dom96 | oh yeah, everyone should be using net now. |
| 17:07:13 | milosn | its basicaly XML frames prefixed with 4 bytes of following XML frame length |
| 17:07:16 | dom96 | Sockets will be deprecated eventually. |
| 17:07:27 | dom96 | I think net supports ssl just like sockets. |
| 17:08:22 | milosn | whats SSL support like? is there a level of abstraction on a top of OpenSSL bindings or i need to know ins/outs of OpenSSL? |
| 17:08:35 | dom96 | Yeah, there is an abstraction. |
| 17:08:39 | milosn | cool |
| 17:08:41 | dom96 | You use newContext and wrapSocket |
| 17:09:05 | * | boydgreenfield joined #nimrod |
| 17:09:36 | * | boydgreenfield quit (Read error: Connection reset by peer) |
| 17:10:24 | * | boydgreenfield joined #nimrod |
| 17:10:49 | milosn | dom96: ill have a look at 'net' |
| 17:11:31 | milosn | ive seen async libs exist as well, just i am worried i dont have the full understanding of how to combine async/threads/db_access in nimrod |
| 17:12:01 | milosn | so looking at bog standard ssl socket with fork() solution :P |
| 17:12:28 | * | milosn wants to build prototype and see how it performs |
| 17:13:32 | EXetoC | will "if x y:" work (command syntax)? |
| 17:14:05 | * | boydgreenfield quit (Read error: Connection reset by peer) |
| 17:14:21 | fowl | prolly |
| 17:15:00 | * | q66 joined #nimrod |
| 17:15:00 | * | q66 quit (Changing host) |
| 17:15:00 | * | q66 joined #nimrod |
| 17:15:06 | * | boydgreenfield joined #nimrod |
| 17:16:19 | * | boydgreenfield quit (Read error: Connection reset by peer) |
| 17:17:02 | * | boydgreenfield joined #nimrod |
| 17:18:03 | EXetoC | yeah. fewer corner cases to remember |
| 17:19:53 | * | boydgreenfield quit (Read error: Connection reset by peer) |
| 17:20:38 | * | boydgreenfield joined #nimrod |
| 17:20:49 | * | boydgreenfield quit (Read error: Connection reset by peer) |
| 17:20:49 | EXetoC | fewer edits too |
| 17:20:51 | milosn | dom96: ive seen you jester project |
| 17:20:59 | milosn | you run any live apps on this? :) |
| 17:21:07 | milosn | your even :D |
| 17:21:44 | * | boydgreenfield joined #nimrod |
| 17:21:55 | * | boydgreenfield quit (Read error: Connection reset by peer) |
| 17:22:37 | dom96 | milosn: the forum runs on jester |
| 17:22:43 | dom96 | milosn: nimbuild too |
| 17:22:43 | * | boydgreenfield joined #nimrod |
| 17:23:08 | milosn | hmm cool |
| 17:23:49 | * | Matthias247 joined #nimrod |
| 17:23:58 | * | boydgreenfield quit (Read error: Connection reset by peer) |
| 17:24:04 | * | hoverbea_ quit () |
| 17:24:42 | * | boydgreenfield joined #nimrod |
| 17:25:10 | milosn | we are php shop at work, but mainly because boss is/was a php guy, so we shit loads of legacy apps ... more and more he does paper work and management these days ... ive been looking for something replace all our php rubbish, and escape the LAMP stack |
| 17:25:14 | milosn | :) |
| 17:25:46 | dom96 | cool. Would be awesome if you used Jester at work. |
| 17:26:23 | milosn | its unlikely to happen any time soon ... i started building an ORM, because first target in replacing PHP is out data access library |
| 17:26:30 | milosn | *our |
| 17:26:42 | * | boydgreenfield quit (Read error: Connection reset by peer) |
| 17:27:00 | milosn | if i can prove that we could have more performant and type checked, to a degree, data access layer |
| 17:27:19 | milosn | ... it might happen |
| 17:27:20 | milosn | :) |
| 17:27:32 | * | boydgreenfield joined #nimrod |
| 17:27:44 | * | boydgreenfield quit (Read error: Connection reset by peer) |
| 17:28:50 | * | boydgreenfield joined #nimrod |
| 17:29:06 | * | boydgreenfield quit (Read error: Connection reset by peer) |
| 17:29:41 | * | kunev joined #nimrod |
| 17:29:56 | * | boydgreenfield joined #nimrod |
| 17:30:06 | * | boydgreenfield quit (Read error: Connection reset by peer) |
| 17:30:09 | milosn | plus apart from running small compiled "scripts" ... i have no experience of quality of nimrod really ... GC etc. |
| 17:30:18 | milosn | :) |
| 17:30:50 | * | boydgreenfield joined #nimrod |
| 17:31:32 | * | boydgreenfield quit (Read error: Connection reset by peer) |
| 17:32:32 | * | boydgreenfield joined #nimrod |
| 17:32:58 | * | boydgreenfield quit (Read error: Connection reset by peer) |
| 17:33:23 | * | BitPuffin joined #nimrod |
| 17:33:50 | * | boydgreenfield joined #nimrod |
| 17:34:56 | * | springbok quit (Read error: Connection reset by peer) |
| 17:35:08 | * | boydgreenfield quit (Read error: Connection reset by peer) |
| 17:37:08 | * | boydgreenfield joined #nimrod |
| 17:41:18 | * | boydgreenfield quit (Read error: Connection reset by peer) |
| 17:42:17 | * | boydgreenfield joined #nimrod |
| 17:42:46 | * | boydgreenfield quit (Read error: Connection reset by peer) |
| 17:44:31 | * | boydgreenfield joined #nimrod |
| 17:44:46 | * | boydgreenfield quit (Read error: Connection reset by peer) |
| 17:46:03 | * | boydgreenfield joined #nimrod |
| 17:46:18 | * | boydgreenfield quit (Read error: Connection reset by peer) |
| 18:02:10 | * | brson joined #nimrod |
| 18:05:29 | EXetoC | can json.`%` be used like this %{} for constructing an empty node? |
| 18:06:59 | EXetoC | I guess not. I'm just trying to figure out a nice way to construct an empty bson document |
| 18:08:08 | EXetoC | but "newBson()" isn't so bad. especially not in conjunction with default args |
| 18:09:04 | * | wan joined #nimrod |
| 18:09:56 | fowl | EXetoC, try %{:} |
| 18:11:17 | EXetoC | gah. I'm pretty sure I tried that at one point |
| 18:11:45 | EXetoC | thanks |
| 18:11:58 | EXetoC | %{:}-< |
| 18:12:15 | njoejoe | dom96: I am trying to figure out how to debug your downforeveryone: https://gist.github.com/dom96/49427cd0a9c8da47fff3 it causes the CPU to spin once it is accessed. Any idea how to approach it? Do you see the same thing? I'm on linux. |
| 18:13:06 | dom96 | njoejoe: You mean it causes 100% CPU usage? |
| 18:15:12 | njoejoe | yes (on one core) |
| 18:15:13 | EXetoC | oh yeah I mentioned that, but of course I didn't officially report it |
| 18:16:15 | dom96 | hrm |
| 18:16:53 | dom96 | The only way that can happen is if epoll returns immediately all the time. |
| 18:17:16 | dom96 | Most likely situation that can happen is if the socket wants write events |
| 18:18:00 | dom96 | So perhaps there is a bug somewhere which asks for write notifications when it doesn't need them |
| 18:19:31 | dom96 | if you can't figure out how to debug it then please report it |
| 18:20:34 | dom96 | But if you want to try then take a look at the selectors module |
| 18:21:18 | dom96 | you can uncomment that echo on line 112 |
| 18:23:27 | Araq | hey, anything important? |
| 18:35:36 | EXetoC | the C++ issues? :p |
| 18:35:59 | * | hoverbear joined #nimrod |
| 18:36:09 | EXetoC | maybe you can provide me with some hints. I'm very impatient when it comes to these things |
| 18:36:12 | EXetoC | hoverbear, hola |
| 18:36:39 | * | hoverbea_ joined #nimrod |
| 18:38:43 | EXetoC | I'll create a PR with the initial changes, and then re-post the output of the remaining errors |
| 18:40:14 | * | hoverbear quit (Ping timeout: 240 seconds) |
| 18:41:29 | dom96 | yay hoverbea_ is still here. |
| 18:42:00 | Araq | EXetoC: ok do that |
| 18:42:57 | EXetoC | will do in an hour |
| 18:46:41 | * | PortableEXetoC2 joined #nimrod |
| 18:46:56 | * | Demos joined #nimrod |
| 18:47:43 | * | hoverbea_ quit () |
| 18:51:23 | njoejoe | dom96: ok, hopefully i can do it today. btw, i don't get the spinning when using wget, only when i use the browser for localhost:8080/http://www.google.com/ and now i see the browser is asking for favicon.ico. so Replying with: EOS: It's down :( for favicon.ico causes the spin. wget http://localhost:8080/noexist also causes the spin... |
| 18:58:04 | * | hoverbear joined #nimrod |
| 18:58:07 | dom96 | njoejoe: interesting. |
| 18:58:42 | PortableEXetoC2 | Exception-free signalling with these high level interfaces would be nice |
| 18:58:56 | PortableEXetoC2 | I think i said that before |
| 18:59:38 | dom96 | PortableEXetoC2: And how should that work? |
| 19:00:36 | dom96 | njoejoe: I would think that for such a URL getAddrInfo would fail. Not sure why it would cause high CPU usage. |
| 19:00:49 | PortableEXetoC2 | Optionally that is |
| 19:01:13 | PortableEXetoC2 | I dont know but it's useful for tasks like these |
| 19:01:27 | dom96 | PortableEXetoC2: You can check for exceptions in futures manually. |
| 19:01:28 | PortableEXetoC2 | Where failures arent exactly exceptional |
| 19:01:54 | dom96 | var fut = recvLine(sock) |
| 19:02:22 | PortableEXetoC2 | There you go. So maybe the crawler should do that then |
| 19:02:24 | dom96 | echo fut.failed # true |
| 19:02:27 | dom96 | If fut failed. |
| 19:02:28 | PortableEXetoC2 | And connecting? |
| 19:02:52 | dom96 | That doesn't allow you to use 'await' |
| 19:04:21 | PortableEXetoC2 | Async + return codes |
| 19:06:00 | PortableEXetoC2 | Just need to chuck more procs at it, right |
| 19:10:18 | PortableEXetoC2 | Ive been thinking about how to make error signalling through return codes really transparent using metaprogramming |
| 19:12:36 | PortableEXetoC2 | And eventually you end up manipulating the whole tree i guess |
| 19:17:01 | PortableEXetoC2 | If you replace exception handling with semi-transparent error code propagation :p |
| 19:18:17 | Araq | I recently had the idea that 'errno' is not that bad, if it is an fact a seq of errors that's appended to |
| 19:18:40 | Araq | so you can check this list whenever you feel like it |
| 19:19:02 | Araq | but at program exit every error that hasn't been popped is reported |
| 19:25:52 | * | hoverbear quit () |
| 19:26:09 | EXetoC | interesting |
| 19:27:44 | EXetoC | I might not mind having to explicitly discard error codes though. the syntax doesn't have to be very intrusive I guess |
| 19:29:08 | * | brson quit (Ping timeout: 252 seconds) |
| 19:30:07 | * | hoverbear joined #nimrod |
| 19:30:28 | Araq | the error would include the stack trace in debug mode, of course |
| 19:31:10 | Araq | might be really handy for a web-service, so exceptions that fuck your control flow and easy error logging |
| 19:33:24 | Araq | *no exceptions |
| 19:35:56 | EXetoC | and sometimes you don't want to wait >9000ms before you can keep going of course |
| 19:36:56 | Araq | huh? what do you mean? waiting 9s? |
| 19:37:28 | dom96 | bbl |
| 19:38:08 | EXetoC | well, I guess you won't be rolling back very much anyway in that case, so throwing might not have a big impact |
| 19:39:29 | EXetoC | but do you think any of this would require additional language features? for example, what about my feature request for allowing exceptions to be raised in blocks that are used as expressions? |
| 19:40:24 | Araq | one point of this approach is to not require any language features |
| 19:40:51 | Araq | and your feature request makes no sense in this context: |
| 19:40:57 | Araq | let foo = case |
| 19:41:04 | * | hoverbear quit () |
| 19:41:09 | Araq | of valueA: 1 |
| 19:41:13 | Araq | of valueB: 2 |
| 19:41:35 | Araq | else: (logError("invalid value"); 2) |
| 19:41:55 | Araq | note that there is no 'raise', but 'logError' so control flow continues |
| 19:42:05 | Araq | and you really need to provide a value |
| 19:42:56 | Araq | the problem with this errorSeq approach is that it sucks for developing libraries; open(f); f.write(...); f.close() all need to check for some invalid file handle 'f' and become no-ops |
| 19:43:30 | Araq | though that's what the OS already has to do ... |
| 19:43:39 | Araq | so maybe it's not bad at all |
| 19:44:08 | fowl | why the hate on exceptions |
| 19:45:01 | Araq | fowl: they are quite expensive for async |
| 19:45:40 | Araq | iirc dom96 benchmarked 5400 vs 5300 req/s for the exception unsafe vs exception safe handling of http requests or something |
| 19:45:54 | EXetoC | Araq, and I guess you just specify some dummy value when rolling back too. |
| 19:46:08 | fowl | so the cost is 100 reqs per second |
| 19:46:28 | EXetoC | but several languages let you roll back and not specify a value to be returned |
| 19:46:56 | EXetoC | ok |
| 19:47:07 | * | kiwi1423 joined #nimrod |
| 19:47:08 | EXetoC | still, this is an interesting approach |
| 19:47:15 | kiwi1423 | Q: What’s the proper way to use built-in GCC functions w/ Nimrod? |
| 19:47:30 | kiwi1423 | (e.g., __popcountl or __builtin_prefetch) |
| 19:48:13 | Araq | lib/system/atomics.nim does it properly, kiwi1423 |
| 19:49:44 | kiwi1423 | Thx! |
| 19:49:51 | Araq | fowl: GPUs also don't support exception handling in any way, hence I'm thinking about alternatives |
| 19:53:41 | Demos | how often do you do things on the GPU that should be raising exceptions? |
| 19:54:26 | EXetoC | Araq, what about rolling back by default using this approach? |
| 19:54:52 | Araq | EXetoC: what do you want to roll back? IO? ;-) |
| 19:55:14 | EXetoC | obviously we should replace the whole exception mechanism with this, and without much of an impact on the syntax :p |
| 19:55:35 | EXetoC | Araq, no, I mean rolling back in the stack tree |
| 19:55:54 | EXetoC | as in what happens when exceptions are raised |
| 19:57:11 | Araq | well it's added to the error seq |
| 19:57:16 | Araq | not sure what you mean |
| 19:58:07 | EXetoC | then you're logging by default, rather than rolling back and possibly reaching the main function if not caught at an earlier stage |
| 20:02:39 | Araq | Demos: well I want to run anything on the GPU :P |
| 20:06:49 | * | kiwi1423 quit (Quit: http://www.kiwiirc.com/ - A hand crafted IRC client) |
| 20:08:21 | EXetoC | you're not doing anything by default, really. you need at least a pragma to make any of these steps implicit |
| 20:08:42 | EXetoC | fortunately you can just push one at the top of the module. yay |
| 20:15:29 | * | PortableEXetoC2 quit (Quit: cake) |
| 20:26:31 | * | brson joined #nimrod |
| 20:26:59 | * | kunev quit (Quit: Lost terminal) |
| 20:27:15 | * | hoverbear joined #nimrod |
| 20:36:30 | * | uggedal left #nimrod (#nimrod) |
| 20:44:26 | hoverbear | dom96 \o/ |
| 20:53:07 | * | Matthias247 quit (Read error: Connection reset by peer) |
| 21:00:32 | * | nande quit (Remote host closed the connection) |
| 21:01:19 | EXetoC | but isn't the problem then that some template instantiation might modify the control, which makes automatic unwinding provided by a library difficult |
| 21:03:05 | EXetoC | no, the language already takes care of invoking ctors etc anyway |
| 21:05:34 | * | Xuerian quit (Quit: No Ping reply in 180 seconds.) |
| 21:05:58 | dom96 | Araq: IIRC the difference was smaller than that. Like 10, and it's possible that it was a slight anomaly, i.e. not related to exceptions. |
| 21:06:32 | EXetoC | that sure is low overhead |
| 21:08:13 | * | Xuerian joined #nimrod |
| 21:08:58 | EXetoC | what does he even mean with "exception unsafe" though? relying solely on return codes? |
| 21:10:16 | EXetoC | I mean safe (no-throw?) |
| 21:13:07 | EXetoC | Araq, ok. we got to boot on the GPU some time |
| 21:30:32 | Araq | EXetoC: "exception unsafe" here means "doesn't work with exceptions but luckily no was thrown" |
| 21:34:14 | EXetoC | that's an entirely different story |
| 21:34:41 | Varriount|Mobile | kiw |
| 21:34:52 | EXetoC | kiwi? |
| 21:35:17 | EXetoC | kiwi1423? he left |
| 21:35:19 | Varriount|Mobile | yeah, I meant to message him/her |
| 21:44:57 | renesac | Araq, dom96: don't ditch exceptions because 0,1% performance loss on one benchmark, let's not follow Go |
| 21:46:31 | dom96 | renesac: Of course not. I love exceptions. |
| 21:49:20 | renesac | :) |
| 21:51:09 | Araq | just to be clear: no, nimrod won't lose its awesome tracked exceptions that put Java to shame |
| 21:51:16 | EXetoC | again, these measurements were for un-raised exceptions |
| 21:52:25 | Araq | the compiler itself uses exceptions, for control flow even |
| 21:52:39 | EXetoC | and that's not what we were discussing. anyway, you can just keep the original interface, and extend it if it really matters |
| 21:53:46 | renesac | right, and I hope one will be able to use them with async too, like in C#, right? |
| 21:54:25 | EXetoC | yes |
| 21:54:33 | Araq | renesac: right, I think they already work |
| 21:54:50 | Araq | dom96: please correct me if I'm wrong |
| 21:54:57 | EXetoC | you can't get the current exception yet in an exception handler |
| 21:55:11 | renesac | ok, then I didn't understood the discussion above |
| 21:55:17 | renesac | but no problem then |
| 21:55:33 | Araq | renesac: but but but ... GPUs! |
| 21:55:45 | renesac | it looked like a regression to checking error codes was being ... |
| 21:55:49 | EXetoC | but are raised exceptions really ~9000 times slower than the semantically equivalent code that uses return codes? |
| 21:56:13 | renesac | Araq, you don't want a 'if' on gpu either... |
| 21:56:16 | Araq | EXetoC: who said that? |
| 21:56:25 | renesac | after each call |
| 21:56:46 | Araq | renesac: my workaround doesn't require this ;-) |
| 21:56:49 | EXetoC | Araq, some people apparently |
| 21:56:56 | EXetoC | that's an exaggeration of course |
| 21:57:18 | njoejoe | dom96: strace ./downforeveryone ; other shell: wget http://localhost:8080/asdf ; and get zillion of these: epoll_ctl(4, EPOLL_CTL_MOD, 6, {EPOLLRDHUP, {u32=6, u64=6}}) = 0 ; epoll_wait(4, {{EPOLLHUP, {u32=6, u64=6}}}, 64, 500) = 1 |
| 21:57:18 | njoejoe | |
| 21:57:42 | Araq | EXetoC: 'throw' is dead slow on Java since it needs to build the stack trace |
| 21:58:07 | renesac | I would wait someone with experience with gpu programming try to use nimrod and say what they need... |
| 21:58:14 | Araq | but that only affects Java/JVM |
| 21:59:00 | dom96 | njoejoe: That doesn't really tell me much. Except it perhaps reaffirms what I said previously. |
| 21:59:26 | renesac | http://en.wikipedia.org/wiki/You_aren't_gonna_need_it |
| 21:59:38 | njoejoe | dom96: ok. |
| 22:00:46 | Araq | renesac: "If I had asked people what they wanted, they would have said faster horses." |
| 22:01:00 | renesac | "Until the feature is actually needed, it is difficult to fully define what it should do and to test it." |
| 22:01:10 | renesac | "Any new feature imposes constraints on what can be done in the future, so an unnecessary feature may preclude needed features from being added in the future." |
| 22:01:13 | * | BitPuffin quit (Quit: WeeChat 0.4.3) |
| 22:02:28 | * | BitPuffin joined #nimrod |
| 22:02:36 | EXetoC | Araq, but you did mention performance when you first responded to me, right? in which case some might need glorified return codes, but only in rare cases |
| 22:03:09 | Araq | EXetoC: sure |
| 22:04:44 | * | def- joined #nimrod |
| 22:05:13 | Araq | renesac: I know about YAGNI, thanks |
| 22:05:15 | Araq | hi def- welcome |
| 22:05:37 | def- | Hi |
| 22:07:11 | renesac | PyCUDA exposes only exceptions to python programmers: Automatic Error Checking. All CUDA errors are automatically translated into Python exceptions. |
| 22:07:18 | renesac | aparently |
| 22:07:57 | Araq | yeah looks like they don't need it :P |
| 22:08:32 | njoejoe | dom96: btw, i uncommented line 112 of selectors.nim and this is what is output https://gist.github.com/jots/c25729dda2d7d0894322 still trying to narrow in on it (because i really want async to work...) |
| 22:09:45 | dom96 | njoejoe: looks fine to me. |
| 22:09:59 | dom96 | njoejoe: I'll test it now |
| 22:12:04 | njoejoe | i don't think it's fine because it is generating tons of those messages when you give it a bad url. when you give it a good url, it does its little bit of work and waits, not spinning like crazy with Epoll: 6 {} {} |
| 22:15:48 | dom96 | njoejoe: you're right. It's using 100% even for correct URLs though. |
| 22:18:52 | * | boydgreenfield joined #nimrod |
| 22:20:11 | njoejoe | i haven't seen it with correct urls yet. but if you're using a browser it may be asking for favicon.ico behind the scenes which is an invalid url. |
| 22:20:18 | Varriount|Mobile | Hi boydgreenfield, any progress with RST docgen? |
| 22:20:29 | dom96 | njoejoe: I'm not. |
| 22:21:01 | * | Varriount-Mobile joined #nimrod |
| 22:21:25 | boydgreenfield | Varriount: No – got distracted by some other items today. But will update if I find a pattern that seems sensible / useful. |
| 22:23:46 | EXetoC | "if (!()) goto LA65;" crap |
| 22:23:46 | * | Varriount-Mobile quit (Read error: Connection reset by peer) |
| 22:24:42 | * | Varriount|Mobile quit (Ping timeout: 240 seconds) |
| 22:29:33 | * | Kazimuth joined #nimrod |
| 22:31:23 | * | superfunc joined #nimrod |
| 22:31:48 | superfunc | hey everybody |
| 22:32:37 | EXetoC | morning |
| 22:34:57 | superfunc | I have a API design question, if you don't mind lending your opinion |
| 22:36:41 | EXetoC | disclaimer: I'm wrong roughly 50% of the time |
| 22:36:44 | EXetoC | ok, go on |
| 22:40:25 | superfunc | would you prefer a highly parameterized type or multiple distinct types. For example. Option A would result in client code looking like this... var g = Graph[int](directed: true, multiedge: true).. whereas option B would look like... var g = DirectedMultigraph[int](..) |
| 22:40:37 | * | brson quit (Quit: leaving) |
| 22:44:09 | Araq | superfunc: let the implementation decide. I can imagine the code works well with Option A and option B will become a pita |
| 22:44:25 | Araq | where you have lots of procs and they all delegate to common templates |
| 22:45:24 | Araq | EXetoC: cgen.nim, lin 507 |
| 22:45:34 | Araq | make the check: |
| 22:46:06 | Araq | if sfVolatile in s.flags or (p.nestedTryStmts.len > 0 and gCmd != cmdCompileToCpp): |
| 22:48:40 | superfunc | Araq: Thanks. For option B, I was considering having a typeclass imposing some basic behavior so I wouldn't have to write too many procs |
| 22:50:55 | dom96 | njoejoe: You can change line 32 of the selectors module to 'when false' as a workaround |
| 22:51:19 | dom96 | njoejoe: Please report this on github, it's a bit of a challenge to fix and I don't have the time right now to do it. |
| 22:51:30 | EXetoC | it's mostly about having clearly defined interfaces, but I guess it might result in fewer procs in some cases |
| 22:52:55 | EXetoC | and type classes work, but not those that are so-called user-defined don't really |
| 22:53:04 | EXetoC | some bugs need to be resolved |
| 22:54:16 | Araq | Error: 0..15 not disjoint from 14..30 |
| 22:54:23 | Araq | is that even correct english? |
| 22:54:38 | EXetoC | the former is just T|U|V, while the latter involves the 'generic' keyword |
| 22:55:56 | * | askatasuna quit (Ping timeout: 252 seconds) |
| 22:59:13 | * | yogin joined #nimrod |
| 22:59:21 | * | yogin quit (Remote host closed the connection) |
| 22:59:41 | * | brihat joined #nimrod |
| 23:00:39 | Kazimuth | is it possible to tell koch to build with clang? Just out of curiosity |
| 23:02:22 | Kazimuth | ookay, nimrod.cfg, that looks promising |
| 23:03:43 | Araq | koch boot --cc:clang |
| 23:03:53 | Araq | (I think) |
| 23:04:55 | EXetoC | oh |
| 23:05:48 | EXetoC | doesn't say that in the help anyway |
| 23:06:01 | * | darkf joined #nimrod |
| 23:06:28 | * | io2 quit (Quit: ...take irc away, what are you? genius, billionaire, playboy, philanthropist) |
| 23:06:39 | Araq | why should it? it passes the options to "nimrod". should koch document everything then that nimrod supports? |
| 23:07:44 | EXetoC | I don't think it's mentioned in the help for either |
| 23:08:19 | Araq | boot [options] bootstraps with given command line options |
| 23:08:50 | Kazimuth | ah, that makes sense |
| 23:08:53 | EXetoC | I meant cc |
| 23:08:58 | Araq | oh |
| 23:28:10 | EXetoC | the volatile error appears to remain |
| 23:28:32 | Araq | EXetoC: but there should be fewer |
| 23:33:00 | * | hoverbear quit () |
| 23:45:30 | * | superfunc quit (Ping timeout: 265 seconds) |
| 23:46:34 | Kazimuth | woo, glut works! |
| 23:46:41 | Kazimuth | ...I should do homework |
| 23:46:46 | Kazimuth | thanks for the help! |
| 23:46:49 | * | Kazimuth quit (Quit: gas leak) |
| 23:46:50 | * | askatasuna joined #nimrod |
| 23:49:35 | Araq | good night |
| 23:52:04 | NimBot | Araq/Nimrod new_spawn c43e8df Araq [+0 ±10 -0]: progress for the 'parallel' statement |
| 23:53:15 | * | brson joined #nimrod |
