<< 12-05-2016 >>
00:01:38*bozaloshtsh quit (Ping timeout: 244 seconds)
00:06:22*yglukhov joined #nim
00:10:33*yglukhov quit (Ping timeout: 240 seconds)
00:33:11*libman quit (Remote host closed the connection)
00:37:57*enthus1ast quit (Read error: Connection reset by peer)
00:38:49*enthus1ast joined #nim
00:42:50*yglukhov joined #nim
00:47:57*yglukhov quit (Ping timeout: 276 seconds)
01:04:00*GangstaCat quit (Quit: Leaving)
01:19:11*yglukhov joined #nim
01:19:31*GangstaCat joined #nim
01:23:13*bozaloshtsh joined #nim
01:23:51*yglukhov quit (Ping timeout: 265 seconds)
01:39:24*vendethiel quit (Ping timeout: 250 seconds)
01:39:33*vendethiel joined #nim
01:43:26*yglukhov joined #nim
01:44:11*brson quit (Quit: leaving)
01:48:24*yglukhov quit (Ping timeout: 276 seconds)
02:19:29*yglukhov joined #nim
02:24:09*yglukhov quit (Ping timeout: 276 seconds)
02:55:39*yglukhov joined #nim
03:00:33*yglukhov quit (Ping timeout: 276 seconds)
03:20:04*yglukhov joined #nim
03:24:27*yglukhov quit (Ping timeout: 260 seconds)
03:52:52*perturbation joined #nim
03:56:22*yglukhov joined #nim
03:57:02perturbationanyone else using Nim for machine learning / data science type tasks?
03:57:27perturbationread through http://rnduja.github.io/2015/10/21/scientific-nim/ and am using the linalg library on a small-scale project
03:58:59perturbationbut would be great if there are other libraries out there that I've overlooked ;)
03:59:23perturbationI saw there's a libsvm wrapper in the stdlib but haven't had a chance to play with it yet
04:00:30*yglukhov quit (Ping timeout: 246 seconds)
04:31:42*brson joined #nim
04:32:37*yglukhov joined #nim
04:37:24*yglukhov quit (Ping timeout: 276 seconds)
04:56:56*yglukhov joined #nim
04:58:44*silven quit (Ping timeout: 250 seconds)
04:59:57*silven joined #nim
05:01:17*yglukhov quit (Ping timeout: 260 seconds)
05:05:54tautologicoperturbation: I'm new to nim, but thanks for the link, it's right up my alley
05:06:29perturbation:)
05:06:54*GangstaCat quit (Ping timeout: 260 seconds)
05:07:39tautologicoI may have interest in doing machine learning and related stuff in nim, still evaluating
05:08:04tautologicofor data analysis I think a good dataframe library is very convenient
05:15:46perturbationagreed - I really miss data.table from R
05:19:57*GangstaCat joined #nim
05:35:44*perturbation quit ()
05:45:06*endragor joined #nim
05:56:57*gokr joined #nim
05:58:36*vendethiel- joined #nim
05:58:39*vendethiel quit (Ping timeout: 244 seconds)
06:00:25*zaquest quit (Quit: Leaving)
06:16:06*bozaloshtsh quit (Changing host)
06:16:06*bozaloshtsh joined #nim
06:17:28*endragor_ joined #nim
06:20:37*endragor quit (Ping timeout: 260 seconds)
06:21:34*kier quit (Ping timeout: 252 seconds)
06:23:25*kier joined #nim
06:32:39*endragor_ quit (Remote host closed the connection)
06:33:18*endragor joined #nim
06:33:56*brson quit (Ping timeout: 276 seconds)
06:40:35*brson joined #nim
06:41:13*gokr left #nim (#nim)
06:44:23*mahlon quit (Read error: Connection reset by peer)
06:46:01*mahlon joined #nim
06:48:35*yglukhov joined #nim
06:53:51*jackv quit (Ping timeout: 264 seconds)
06:57:20*Jesin quit (Ping timeout: 276 seconds)
06:58:25*jackv joined #nim
07:06:55*Jesin joined #nim
07:58:39*Trustable joined #nim
08:35:38*Arrrr joined #nim
08:45:36*PMunch joined #nim
09:00:00*brson quit (Ping timeout: 244 seconds)
09:06:35*Demon_Fox quit (Quit: Leaving)
09:08:10*brson joined #nim
09:20:19*enthus1ast quit (Ping timeout: 252 seconds)
09:26:07yglukhovdom96: hey, found the issue, i mentioned yesterday. the problem is WSArecv
09:29:59*brson quit (Ping timeout: 260 seconds)
09:32:40*arnetheduck joined #nim
09:37:42*brson joined #nim
09:54:58cheatfateyglukhov, interesting
09:55:20cheatfateproblem in windows api function? :)
09:55:29yglukhovin its usage
09:55:41yglukhovare you an expert in winsocks by any chance? =)
09:55:50cheatfateexplain
09:56:14cheatfateyglukhov, lets found your bug
09:56:21cheatfatefind*
09:57:06yglukhovhttps://github.com/nim-lang/Nim/blob/devel/lib/pure/asyncdispatch.nim#L767
09:57:22yglukhovwsasend may return 0 and bytesReceived == 0
09:57:44cheatfateyou mean wsarecv
09:57:50yglukhovin such case setting realSize to size is extremely wrong.
09:58:03yglukhovyeah, wsarecv
09:58:39yglukhovso i fixed it by not calling complete in case of bytesReceived == 0
09:59:10yglukhovin such way, socket completion handler will later be called with nice values, and complete the future
09:59:40cheatfateyglukhov, you are right this is weird part of code...
09:59:47yglukhovbut, msdn says, that it is recommended to pass NULL instead of bytesReceived in case of overlapped socket
09:59:53cheatfatebut i never fall to this branch
10:00:44yglukhovwe can reproduce it by sending around 10000 requests to mongo in a row.
10:01:09yglukhovunder "normal" load it doesnt reproduce
10:02:02cheatfateIf an overlapped operation completes immediately, WSARecv returns a value of zero and the lpNumberOfBytesRecvd parameter is updated with the number of bytes received and the flag bits indicated by the lpFlags parameter are also updated.
10:02:42cheatfateOverlapped Socket I/O
10:02:42cheatfateIf an overlapped operation completes immediately, WSARecv returns a value of zero and the lpNumberOfBytesRecvd parameter is updated with the number of bytes received and the flag bits indicated by the lpFlags parameter are also updated.
10:03:28cheatfatethere many texts in wsarecv documentation which are contradict each other
10:04:37cheatfateand there 2 types of overlapped io can be used... overlapped with event and overlapped without event... asyncdispatch uses overlapped without event
10:06:25cheatfatei think simple usage of lpNumberOfBytesRecvd result could be ok but needs testing
10:06:52cheatfateand we always can just delete this branch because completion routine will be called anyway
10:09:41cheatfateyglukhov, try to test it with simple usage of bytesReceived
10:12:33*brson quit (Ping timeout: 240 seconds)
10:14:14cheatfateand i think when wsarecv returns 0 and bytesReceived == 0 then peer was disconnected...
10:19:18*ephja joined #nim
10:29:54*brson joined #nim
10:30:09*brson quit (Read error: Connection reset by peer)
10:30:37dom96yglukhov: Glad you found the problem. Think you could create a PR with the fix? :)
10:32:00dom96Sucks that there are edge cases like this, I remember not being sure what to do in that case but whatever way I reproduced it must have been slightly different again.
10:35:18*elrood joined #nim
10:39:37yglukhovdom96: also there seems to be a lot of extra copying. what do you think if we preallocate the result string with `size`, and put it into the Future immediately, and init buf for wsarecv with addr to it?
10:40:15dom96that should be fine
10:44:56*gokr joined #nim
10:45:33*coffeepot joined #nim
10:49:03*federico3 quit (Quit: WeeChat 1.1.1)
10:49:40*federico3 joined #nim
10:52:17coffeepothey guys, just wondering will the GitHub pricing changes affect Nim?
10:54:19ArrrrIsn't that for private repos?
10:55:12coffeepotyes I think so, so no then? :)
10:58:26dom96nope
10:58:53coffeepot\o/
10:59:50coffeepotbtw dom96, been reading your book. Fantastic stuff! Really looking forward to the metaprogramming chapter, but well done on achieving so much already on it
11:00:59*federico3 quit (Quit: WeeChat 1.1.1)
11:03:17*federico3 joined #nim
11:11:00dom96coffeepot: Thanks :)
11:16:09*BitPuffin joined #nim
11:22:00*MyMind joined #nim
11:24:29*sembei quit (Ping timeout: 276 seconds)
11:25:42*BitPuffin quit (Ping timeout: 244 seconds)
11:27:22*enthus1ast joined #nim
12:49:00*fastrom joined #nim
13:08:11*fastrom quit (Quit: Leaving.)
13:17:28*brson joined #nim
13:20:59*GangstaCat quit (Ping timeout: 260 seconds)
13:21:19*brson quit (Client Quit)
13:21:35*brson joined #nim
13:26:41*nsf quit (Ping timeout: 276 seconds)
13:26:43*GangstaCat joined #nim
13:54:18*vendethiel joined #nim
13:55:19*vendethiel- quit (Ping timeout: 252 seconds)
14:04:57*aziz_ joined #nim
14:13:11cheatfateyglukhov, you are not fully right when "but, msdn says, that it is recommended to pass NULL instead of bytesReceived in case of overlapped socket", i checked source code of libuv and python's async code and they are using bytesReceived when WSARecv returned 0
14:33:49dom96I just tried to sell #hedgewars on Nim
14:34:15dom96It's currently written in Pascal and they want to rewrite it in Rust (or Ada)
14:34:38dom96They rejected it based on: 1) indentation for blocks 2) GC :(
14:34:50cheatfateyglukhov, of course your variant would work, but it just makes async code a little bit slower
14:37:51ldleworkerdom96: *rolls eyes*
14:39:06cheatfatedom96, looks like nobody likes GC
14:39:39dom96Yeah... well, I tried :\
14:39:52endragorparanoids
14:40:56cheatfateendragor, paranoids because they dont like GC?
14:41:14endragoryes
14:41:54cheatfateendragor, hmm, i dont like GC too
14:43:07endragorGC_disable() is the friend of paranoids :)
14:43:14cheatfateGC just adds some kind of uncontrollable entity to your projects...
14:43:22cheatfateGC_disable() = discard i think
14:43:28endragor--gc:none actually
14:44:12endragorcheatfate: it's not
14:44:14cheatfateendragor, then you need to make stdlib again...
14:44:38endragorGC_disable() + GC_step() works well for me
14:44:42dom96no need to be afraid of the GC
14:44:46dom96it's soft real-time
14:44:53dom96i.e. perfect for games
14:44:55endragorright
14:45:14cheatfateendragor, when i made stress tests of asynchttpserver.nim i really dont understand why `wrk` returns such big deviation of results
14:45:43*pregressive joined #nim
14:46:10cheatfateendragor, when you run C code you will not get results like 0.1ms +/- 10ms
14:46:33dom96cheatfate: have you tried with --gc:none?
14:46:47dom96to see if the deviation disappears?
14:48:12cheatfatedom96, i dont think deviation will disappears just because there will be no memory deallocation so processing of requests will just increasing...
14:48:49dom96cheatfate: Aren't you blaming the deviation on the GC?
14:49:06cheatfateyeah i think GC makes deviation
14:49:16endragorcheatfate: Nim provides enough tools to control that. Just use soft realtime guarantees as mentioned above. the place from which GC is called matters a lot - the deeper the stack is, the longer the scan takes
14:49:26dom96yeah, so to test if you're right, disable the GC and see if the deviation disappears
14:49:32dom96if it does then you're right
14:50:29*libman joined #nim
14:50:39cheatfatedom96, if i disable GC it will just allocates memory all the time without deallocation... on stresss tests OS starts to swapping my memory pages... so deviation will increased too
14:51:14dom96cheatfate: I see. Thought it wouldn't allocate that much memory.
14:52:01cheatfatedom96, SIGSEGV: Illegal storage access. (Attempt to read from nil?) this is result of --gc:none
14:53:09dom96pity
14:53:12cheatfatedom96, https://gist.github.com/cheatfate/9cf957d08d29e3c319c54ce97c6212d1
14:53:37dom96and also strange
14:55:57cheatfateit will be very good if "--gc:none" will start to work,
14:58:11endragortry GC_disable(), too. It will still count references, but won't collect garbage periodically
14:58:16*novavis joined #nim
15:01:45yglukhovcheatfate: could you share a link to some python/libuv code where they process wsarecv?
15:03:02yglukhovWe have to ensure that the buffer is empty because WSARecv will tell us immediately when it was disconnected, even when there is still data in the buffer
15:03:08yglukhovdom96: Are you really sure about this?
15:03:46cheatfateyglukhov, https://github.com/libuv/libuv/blob/b12624c13693c4d29ca84b3556eadc9e9c0936a4/src/win/tcp.c#L520
15:03:54dom96yglukhov: maybe create two separate PRs, I'm not sure but would like to see what you have in mind to make a decision.
15:04:13*brson quit (Read error: Connection reset by peer)
15:04:24*aziz_ quit (Remote host closed the connection)
15:04:28yglukhovtbh, i'm a bit lost. havent found the best solution yet
15:08:21cheatfateyglukhov, could you please just test with `realSize = bytesReceived`
15:11:32cheatfateyglukhov, could you please test this variant: https://gist.github.com/cheatfate/373b7f9b03aa29f60c81e2fce8f38c1b/revisions
15:14:01*brson joined #nim
15:14:36*wuehlmaus quit (Quit: Lost terminal)
15:20:07yglukhovcheatfate: completing future when bytesReceived == 0 doesn't work for me. i've already tested it sometime ago.
15:21:16yglukhovit actually can't work if you think of it. future.complete(0) will be firther treated as eof by higher-level api
15:21:26yglukhov* further
15:23:44yglukhovthe best solution i currently came up with is not completing future in case when bytesReceived == 0. it works. deletion of all the code that handles res == 0 and fully relying on the completion handler also works, but as you say it might be less effective
15:23:54yglukhov* efficient
15:25:24dom96yglukhov: that sounds like a good solution
15:25:25cheatfateyglukhov, if it not works, then you are right and we can't use bytesReceived when WSARecv returned 0
15:26:53yglukhovcheatfate: you mean ((when WSARecv returned 0) and (when bytesReceived == 0))?
15:27:19cheatfateyglukhov, wait i will patch my gist properly
15:27:28yglukhovkk, afk 10 min
15:34:08*Jesin quit (Quit: Leaving)
15:40:01cheatfateyglukhov, could you please point me exact branch which you are suspect?
15:40:26*pecan joined #nim
15:41:47dom96cheatfate: I think he mentioned https://github.com/nim-lang/Nim/blob/devel/lib/pure/asyncdispatch.nim#L767 this morning
15:42:41cheatfatedom96, in my stress tests of asynchttpserver.nim on windows this works fine and causes deep recursion problems
15:43:29yglukhovcheatfate: do you enter this specific line with bytesReceived == 0?
15:44:08yglukhov"From my tests bytesReceived isn't reliable", dom96, is this your comment?
15:44:19dom96yglukhov: yep
15:44:40yglukhovis it unreliable in the same way im talking about?
15:44:42cheatfateyglukhov, https://github.com/nim-lang/Nim/blob/devel/lib/pure/asyncdispatch.nim#L759 this line causes problem?
15:45:13cheatfatebecause in this branch i dont like only "and dataBuf.buf[0] == '\0'"
15:45:20dom96yglukhov: I recall that it gave 0 sometimes when data was actually received.
15:45:59dom96is that what's happening in your case?
15:46:09yglukhovdom96: yep.
15:46:14yglukhovcheatfate: nope.
15:46:24yglukhovcheatfate: https://github.com/nim-lang/Nim/blob/devel/lib/pure/asyncdispatch.nim#L767
15:47:17yglukhovnimongo uses buffered async sockets, and the buffer is not zeroed before reading. so the buffer[0] == 0 check might not work in this case
15:48:00dom96yglukhov: but I guess `size` isn't the amount that is received either?
15:48:27yglukhovdom96, no, of course.
15:49:15yglukhovactually, i also don't like the "and dataBuf.buf[0] == '\0'". looks really hacky.
15:49:15dom96ahh, but then you get the real amount that is received from IOCP?
15:49:21yglukhovyup
15:50:16*fastrom joined #nim
15:50:50cheatfatei think we just need to remove " and dataBuf.buf[0] == '\0'":
15:51:05cheatfateand then we can do realSize = bytesReceived
15:51:11yglukhovdom96, cheatfate i would vote for removing all of this branch. what do you say?
15:51:53yglukhovand leave the last branch, and only complete the future when bytesReceived != 0
15:52:54yglukhovthe only question i still havent understood is dom96's comment
15:53:00yglukhovWe have to ensure that the buffer is empty because WSARecv will tell
15:53:05yglukhovus immediately when it was disconnected, even when there is still
15:53:10yglukhovdata in the buffer.
15:53:19cheatfateyglukhov, if you remove all "immediate completion" branch you will hide another bug with "deep recursion"
15:53:33*kulelu88 joined #nim
15:53:49yglukhovcheatfate, no, immediate completion will still be there when bytesReceived > 0
15:54:17cheatfatei think we just need to remove only " and dataBuf.buf[0] == '\0'"
15:55:09yglukhovcheatfate, no, because we can't complete when bytesReceived == 0, because it is __unreliable__
15:55:39yglukhovok, let me do the pr
15:56:49*cheatfate quit (Quit: Leaving)
15:57:09*cheatfate joined #nim
15:59:24dom96hrm
16:00:31cheatfatedom96, i think we need to add one more check
16:00:36dom96but then if you remove that branch what will happen if socket is disconnected?
16:00:57yglukhovbtw, proc recv looks a lot like it could reuse proc recvInto
16:01:23dom96I guess the future will simply be completed with 0 later?
16:01:33cheatfatedom96, wait guys
16:01:37yglukhovdom96: in case of disconnection the future will be completed with the handler
16:01:54cheatfateOVERLAPPED structure has a member which can be checked if operation really completed
16:02:39*coffeepot quit (Quit: http://www.kiwiirc.com/ - A hand crafted IRC client)
16:02:40cheatfatethere is a macro https://msdn.microsoft.com/en-us/library/windows/desktop/ms683244(v=vs.85).aspx
16:05:09cheatfatefrom my tests i can say
16:05:41cheatfatein some cases OVERLAPPED.Internal still STATUS_PENDING
16:05:57yglukhovdo you suggest checking for HasOverlappedIoCompleted when wsarecv returns 0?
16:06:01cheatfatei think in such cases bytesReceived is unreliable
16:06:21cheatfateyep
16:06:35dom96msdn docs say that when `WSARecv` returns 0 the "operation completed immediately"
16:06:37yglukhovdo you mean that your tests produce STATUS_PENDING when wsarecv returns 0?
16:06:45cheatfateyglukhov, yep
16:06:51dom96so won't that check be redundant?
16:07:19cheatfatedom96, we are using OVERLAPPED structure without CreateEvent for hEvent member
16:07:21yglukhovcheatfate: does it produce STATUS_PENDING *every time* when wsarecv returns 0?
16:07:57cheatfateyglukhov, i dont sure maybe it always STATUS_PENDING
16:07:58yglukhovcheatfate: can you corellate STATUS_PENDING with bytesReceived?
16:08:57cheatfateyglukhov, please look here https://github.com/libuv/libuv/blob/b12624c13693c4d29ca84b3556eadc9e9c0936a4/src/win/tcp.c#L534
16:09:36cheatfateyglukhov, authors of libuv for some reason using registerwaitforsingleobject with postqueuedcompletionstatus
16:10:51cheatfateyglukhov, they try to do manual call
16:10:58cheatfateyglukhov, and this is wrong too
16:12:19yglukhovcheatfate: so.. do you have a suggestion?
16:13:03cheatfatei want to check STATUS_PENDING of OVERLAPPED.Internal member like HasOverlappedIoCompleted
16:13:31cheatfateand if OVERLAPPED.Internal != STATUS_PENDING: i think to use bytesReceived
16:13:57yglukhovok, can you confirm that you dont get STATUS_PENDING *every* time wsarecv returns 0. because if you do, then it doesnt make much sense
16:14:36cheatfateyglukhov, you need to make tests too, i will make asynchttpserver stress tests and you can make nimongo tests
16:15:00cheatfateto check if ol.Internal != STATUS_PENDING
16:15:06dom96This snippet from the MSDN docs: https://gist.github.com/dom96/e8eb03b4338d06e3dbc5b917e5a5328b
16:15:25dom96Seems to indicate that when WSARecv returns 0 and bytesReceived is 0 that the socket disconnected.
16:15:31yglukhovi'll try. unfortunately it's not that easy, because i don't have windows, and have to bother another guy :(
16:16:25yglukhovdom96, commented on your gist ;)
16:17:15yglukhovmsdn is like a book of laws. go try interpret :D
16:17:55dom96so according to that we shouldn't be using `bytesReceived` at all?
16:18:10dom96"Use NULL for this parameter if the lpOverlapped parameter is not NULL to avoid potentially erroneous results."
16:18:11dom96D:
16:19:26dom96if that's the case then maybe we should just let the completion handler handle the case when WSARecv returns 0?
16:19:28yglukhovthats what i said previously, but cheatfate insists it would not be as efficient. and from my observations (and from yours either) the only erroneous result i've met is 0.
16:20:15dom96I say go for it.
16:20:29dom96If we can demonstrate that it's significantly slower then we can always revert back and think of a better solution
16:20:47yglukhovcheatfate, please agree ;)
16:20:51dom96it's better to be correct and possibly slower than be faster but incorrect.
16:21:04*PMunch quit (Ping timeout: 252 seconds)
16:24:21cheatfateheh, you guyst just to lazy to make some tests i think
16:24:23cheatfate:)
16:24:29dom96as for reusing recvInto, I think that's a good idea.
16:26:56cheatfateand when you move "immediate completion" out, you will hide "deep recursion" bug which must be resolved too
16:32:20dom96That won't hide it, it will remove it, no?
16:36:12yglukhovcheatfate: so how about your test?
16:37:14cheatfateyglukhov, https://gist.github.com/cheatfate/a9e1a258a72ec29e11a415d5917ef4c9
16:38:35cheatfatemy tests in stress test of asynchttpserver.nim gives me that in 217425 requests there was 9 messages "complete with bytesReceived != 0"
16:39:04cheatfateplease try my variant of asynchttpserver
16:39:16cheatfatenot asynchttpserver but asyncdispatch
16:40:19yglukhovcheatfate: with 9 out of 217425 you will get what you're afraid of.
16:40:58yglukhovthat is too much deferred completion
16:42:33cheatfateyglukhov, could you please test my asyncdispatch with nimongo?
16:42:58yglukhovyes, please wait a bit
16:44:06yglukhovwow, you're from Kyiv!
16:44:24yglukhovhave you been to our Nim workshop?
16:45:22cheatfateyglukhov, i missed it
16:45:53yglukhovah, pity. i guess we're gonna do another one, this fall.
16:46:10cheatfateyglukhov, next one i will not miss :)
16:46:23yglukhovbut keep it a secret please, don't tell anyone ;)
16:46:51cheatfateyglukhov, ok
16:47:38yglukhovcan you try my gist meanwhile please? https://gist.github.com/yglukhov/504513e00bff56dd401f38e1522d6cac
16:49:45dom96yglukhov: you guys should livestream the next one if possible :P
16:50:13dom96in fact, i'm considering setting up some sort of virtual meetup
16:50:20yglukhovbut it will likely be not in English
16:51:15cheatfateyglukhov, your variant is working without asserts
16:51:21yglukhovit looks funny when Unkrainians are speaking English at conferences in Ukraine =)
16:52:18cheatfateyglukhov: and your variant also do not hide "deep recursion" bug
16:52:30cheatfateyglukhov, is it works for you?
16:55:36yglukhovcheatfate: it does
16:55:45yglukhovbut.
16:56:12yglukhovreturning to your measurements: 9 out of 217425.
16:57:06dom96yglukhov: even so, would be nice. ~9% of nim-lang.org visitors are from Russia :)
16:58:29yglukhovdon't you think that checking STATUS_PENDING may defer more completions than it should?
16:59:10yglukhovi mean, i understand the overall sanity behind not touching the buffer when STATUS_PENDING. but still...
16:59:37cheatfateyglukhov, there not so many docs about using OVERLAPPED structure with not initialized hEvent member
17:00:19cheatfateyglukhov, and we can know exactly only when we disassemble wsarecv()
17:01:51yglukhovcheatfate, ok, here's another point. 9 out of 217425 say complete with "complete with bytesReceived != 0". but with my gist and with the current impl you would get 217425 immediate completions instead.
17:02:05yglukhovright?
17:02:27yglukhovand my gist is also proven to work experimentally.
17:03:15yglukhovand you said that immediate completions are better than deffered, right?
17:04:04yglukhovdom96: ok, i guess its not really a problem to setup online stream.
17:04:21cheatfateyglukhov, ofc immediate completions are better, but completion routines will be called anyway...
17:04:56cheatfateyglukhov, because api function which modifies windows socket handle to stop notificate iocp is available only starting from vista
17:05:06yglukhovbut the code wating for socket data will not wait for those routines.
17:05:45*brson quit (Quit: leaving)
17:05:55dom96yglukhov: Yep. Plus, even if it's in a different language, it shows that there is interest in the language.
17:06:34dom96Crystal recently streamed a meetup in Brazil
17:07:42cheatfateyglukhov, i have not received 217425 immediate completions... there was only 9 immediate completions because of my check "ol.internal != STATUS_PENDING"
17:07:54yglukhovdom96, have you considered a meetup in GB?
17:07:56cheatfateall other completions was deffered
17:08:27yglukhovcheatfate: i know. but with my gist you would get all of them immediately.
17:08:36dom96yglukhov: I have, I don't really have a place to do it though.
17:09:09yglukhovdom96, oh thats not an excuse ;)
17:09:15dom96lol
17:09:26yglukhovyou could do it even outdoors.
17:09:30dom96You're right.
17:09:45dom96I could also use more time though, that's a better excuse :P
17:10:08yglukhovoh come on
17:10:46cheatfateyglukhov, your gist was not called even once...
17:11:04cheatfateyglukhov, so your patch just make all completions deffered
17:11:23cheatfatein my test
17:11:26yglukhovwait what
17:11:32yglukhovhow could it be
17:12:12cheatfatei have modified your code with just `echo "completed immediately"`
17:12:20cheatfateand did not received any messages
17:13:05yglukhovare you looking at recv or recvInto?
17:14:26cheatfatewait i was wrong i have added to recv only
17:15:15cheatfateyglukhov, yeah your code also was called 9 times :)
17:15:28cheatfatefrom 215337 requests
17:15:57cheatfatethere was only 9 immediate completions
17:16:18cheatfatebut you need to understand this is was done only with "-d:release"
17:16:18yglukhovoh come on. take a look at it. it completes immediately always when bytesReceived != 0.
17:16:36cheatfateyglukhov, my tests cannot be run in debug mode
17:16:51cheatfatei can't simply make "nim c asynchttpserver"
17:17:05cheatfatebecause i there "deep recursion" bug still exists
17:17:33cheatfatethere will be more "immediate completions" in debug variant
17:17:41cheatfatebut in "release" there only 9 :)
17:18:04cheatfatei even dont know how you nimongo works in debug mode
17:18:22cheatfatemaybe you need to run more heavy test
17:18:41cheatfateto receive "deep recursion"
17:19:49yglukhovnevertheless. do you still think that your variant with STATUS_PENDING is better?
17:21:01cheatfateyglukhov, even dont know, my variant is more bulletproof, just because it can handle "disconnections" immediately... your variant make "disconnections" to be deffered
17:21:02*fastrom quit (Quit: Leaving.)
17:22:21cheatfateyglukhov, but the only reason to use my variant is to handle DDoS
17:22:39yglukhovactually we can combine the best of our solutions.
17:23:15yglukhovi could add STATUS_PENDING check in case bytesReceived == 0. and complete immediately if not STATUS_PENDING
17:23:42cheatfateyglukhov, yeah it would be nice
17:25:18cheatfateyglukhov, but i think you need to add "HasOverlappedIoCompleted"
17:25:25cheatfateto winlean and use it
17:26:39cheatfatecurrently "HasOverlappedIoCompleted" is a macro in "winbase.h"
17:26:47cheatfatefor both windows sdk and mingw
17:31:36yglukhovcheatfate: so is it winbase.h i have to include directly or through some other header?
17:32:36cheatfateyglukhov, i think you can simple include "windows.h"
17:32:53cheatfatemingw's windows.h have include <winbase.h>
17:34:11cheatfatemicrosoft sdk's windows.h also have include <winbase.h>
17:38:05yglukhovcheatfate, i'm afraid i will not be able to test my pr anymore, coz our last windows man has already left the office =). could you file the pr please?
17:38:52cheatfateyglukhov, you want me to make pr?
17:38:58yglukhovyup
17:39:32yglukhovwill ya?
17:40:22cheatfateok
17:40:30yglukhovcool, thanks a bunch
17:40:34cheatfatedo you made recv to use recvInto?
17:40:39yglukhovnope
17:41:20yglukhovi guess it can be done later.
17:41:30yglukhovbut feel free of course ;)
17:43:33*enthus1ast quit (Ping timeout: 240 seconds)
17:44:06yglukhovafk 2h
17:48:44*yglukhov quit (Ping timeout: 260 seconds)
17:52:47cheatfatedom96: https://github.com/nim-lang/Nim/blob/devel/lib/pure/asyncdispatch.nim#L686 do you create here one more copy of data?
18:06:25dom96cheatfate: I don't think so
18:06:41*Jesin joined #nim
18:07:01cheatfatedom96, then i dont understand why you are using `$`
18:14:20dom96cheatfate: might just be a left-over from when 'data' was a cstring
18:29:09*Arrrr quit (Quit: WeeChat 1.4)
18:30:49*yglukhov joined #nim
18:32:03cheatfateyglukhov, check it please https://github.com/nim-lang/Nim/pull/4150
18:32:40cheatfateit works fine with asynchttpserver
18:34:17*Jesin quit (Quit: Leaving)
18:35:54*yglukhov quit (Ping timeout: 276 seconds)
18:37:23*Jesin joined #nim
18:51:12*yglukhov joined #nim
19:06:27*novavis quit (Quit: http://www.kiwiirc.com/ - A hand crafted IRC client)
19:29:47Araq_$ on strings is a nop
19:29:53Araq_optimized away, yay
19:37:25*novavis joined #nim
19:38:14*brson joined #nim
19:42:24*enthus1ast joined #nim
19:43:33*ephja quit (Quit: WeeChat 1.5)
19:44:31*novavis quit (Quit: http://www.kiwiirc.com/ - A hand crafted IRC client)
20:02:10*yglukhov_ joined #nim
20:02:10*yglukhov quit (Read error: Connection reset by peer)
20:05:17*yglukhov_ quit (Read error: Connection reset by peer)
20:05:42*yglukhov joined #nim
20:10:26*yglukhov quit (Read error: Connection reset by peer)
20:10:34*yglukhov joined #nim
20:12:14*yglukhov quit (Read error: Connection reset by peer)
20:15:27*novavis joined #nim
20:15:41*novavis quit (Client Quit)
20:20:51*yglukhov joined #nim
20:24:42*zodiak quit (Ping timeout: 260 seconds)
20:27:03libmanAre there any exciting new asynchttpserver benchmarks by any chance?
20:28:16libmanNim's performance in Web Framework benchmarks has been rather disappointing so far, but I see great potential... http://forum.nim-lang.org/t/1152
20:31:14libmanhttps://github.com/nanoant/WebFrameworkBenchmark didn't update in 5 months...
20:36:46*Jesin quit (Quit: Leaving)
20:56:55*zodiak joined #nim
20:57:20dom96libman: afraid not
20:58:13libmanAny future predictions? :P
20:58:51*endragor quit (Read error: Connection reset by peer)
20:59:09*endragor joined #nim
21:02:16*fastrom joined #nim
21:17:15*Demon_Fox joined #nim
21:20:40*vendethiel quit (Quit: q+)
21:26:17*pregressive quit (Remote host closed the connection)
21:31:03*GangstaCat quit (Quit: Leaving)
21:36:59dom96Araq_: why does this fail to compile? https://gist.github.com/dom96/a518108352d0c1249359c6e4807466c6
21:37:42Araq_int doesn't convert to float because both could be 64bits
21:38:42*endragor_ joined #nim
21:42:52*endragor quit (Ping timeout: 260 seconds)
21:43:14*endragor_ quit (Ping timeout: 260 seconds)
21:43:39*GangstaCat joined #nim
21:44:14dom96Araq_: argh, but it's so annoying
21:44:36Araq_converter letItBe(x: int): float = float(x)
21:45:00dom96hehe
21:45:21dom96nice hack, but that just doesn't feel right
21:46:05Araq_well Nim says only conversions which cannot lose precision are implicit
21:46:25Araq_you cannot convert every 64bit int to a 64bit float
21:46:30Araq_so what should we do?
21:46:40dom96why can't we?
21:49:12cheatfatedom96, and why its annoying? just use 5.0 + 5.5 and you will be find
21:49:17cheatfatefine*
21:49:27cheatfateits a good error
21:50:38cheatfatebut ofc maybe i'm too old and dont like floats at all
21:50:41dom96it's annoying because I have to cast manually to an int or float
21:51:40cheatfatedom96, you are trying to make python from nim i think :)
21:55:31dom96I don't think so
21:55:38*gokr quit (Ping timeout: 276 seconds)
21:59:27cheatfatei think now all modern languages trying to return to `strict typing`
21:59:28libmanI second the "make python from nim" proposal. :P
22:00:14libmanoptional strict typing is great, when optional
22:02:44cheatfatelibman, 20 years ago, not all processors even floating point arithmetic inside...
22:02:50cheatfateeven has
22:03:23cheatfateand now you trying to mix something for what processors are not suitable
22:04:44*fastrom quit (Quit: Leaving.)
22:11:40*Jesin joined #nim
22:16:11libmanI'm just brainstorming selling points for Nim. Right now it's neither the fastest nor the most productive language. How can it compete with writing a project in Python and optimizing parts of it in C?
22:16:56Araq_neither Python nor C have a decent static typing system.
22:17:01Araq_neither has good meta programming
22:17:08libmanWhat Nim can be is the language that occupies the broadest width in the middle of the efficiency or productivity tradeoff spectrum.
22:17:38libman*efficiency VS productivity
22:18:31libman(It can also be the copyfree'est language, but I know most people don't care about that.)
22:19:51libmanSo I'm wondering if Nim can take a configurable "why not both" approach on things like implicit casting. Spectrum width FTW!
22:24:04*alexsystemf__ quit (Ping timeout: 260 seconds)
22:25:30*libman left #nim (#nim)
22:25:36*libman joined #nim
22:25:38*alexsystemf__ joined #nim
22:26:35Araq_libman: for me it IS the most productive language. by far. :P
22:28:17tautologicothe tendency now is static typing and compiled
22:29:09tautologicodynamic languages were an artifact from a time when people thought performance didn't matter, because new CPUs would make your code faster in a few months anyway
22:29:18libmanThinking about an objective way to measure dev productivity.
22:37:36libmanReally the package selection matters more than anything else. Writing a nim program to download YouTube videos? In Python, `import youtube_dl` is half the code you need.
22:39:33libmanAnd yet node managed to be very unproductive for me, despite npm packages breeding like rabbits...
22:42:16cheatfatelibman, and when somebody ask you to make program which download youtube video... you give him script and 2 pages of instructions to download and install appropriate version of python, updating pip collection, and installing youtube_dl module... and after all this shit has done, you telling your customer or friend that your program is only working on linux/unix or windows systems... and on windows you need to start it with cmd -> python
22:42:16cheatfateyoutubedownloader.py
22:43:54cheatfatei like python, i like asyncio of python, but i hate people which dont like to make it equal for all systems
22:44:07libmanI used that example because youtube_dl works for multiple video hosting sites and it gets updated every few days to keep track of all the API changes / obfuscations / etc.
22:44:57cheatfatelibman, it doesnt matter because to distribute your software you need to do a big amount of work...
22:45:06*elrood quit (Quit: Leaving)
22:45:07cheatfatewhile you are on python
22:46:19cheatfatelibman, and when you are speaking on speed... asynchttpserver.nim is 10x faster then python's aiohttp
22:47:16libmanI'm not here to evangelize Python, I'm here to convince myself to finally embrace Nim. :)
22:47:48*Trustable quit (Remote host closed the connection)
22:48:28libman(Although, technically, python has that covered with auto-updating executables - https://github.com/cloudmatrix/esky)
22:49:12cheatfatelibman, you can check this code https://gist.github.com/cheatfate/99fa898a9affd370d5cb1225df9f4ca6 it was made just for fun... before nim i can make same thing only in C or ASM
22:52:31*yglukhov quit (Remote host closed the connection)
22:53:03cheatfatenim is just a little bit higher than C and its why i like it
22:54:15libmanWhat does any of that have to do with what I was talking about?
22:55:27libmanA flag to enable implicit casting would widen nim's appeal.
22:56:21cheatfateimplicit casting is a step back in time...
22:57:38cheatfateyeah it makes coding easier but it also adds to code a bunch of chaos which you dont want to control
22:58:54libmanSometimes you just want to tell the compiler: I care about X (ex. productivity in finishing this one-off data converter), not Y (performance, type safety, concurrency, portability, not having access to nuclear launch codes, etc).
23:00:52*yglukhov joined #nim
23:00:53libmanSelling Nim to someone who knows Python and C well is hard.
23:05:50*yglukhov quit (Ping timeout: 276 seconds)
23:06:50cheatfatelibman, i think you also want compiler to which you can tell: I want youtube and facebook video downloader for Linux/Windows/MacOS in one executable package...
23:07:25cheatfateand maybe in next 100 years you will see such thing...
23:07:33libmanThat's a package availability issue, not a compiler issue.
23:08:06*PMunch joined #nim
23:14:23Araq_libman: the real problem is that selling a new language to a happy programmer is hard. IME.
23:14:58Araq_if you think C/Python is a perfectly fine combo, then so be it.
23:15:13Araq_I think C is archaic and I cannot stand Python's dynamic typing.
23:15:27libmanI had to quit Python, mainly because I'm a copyfree fanatic.
23:16:57libmanSo if the Nim community was to ostracize programmers to write non-copyfree "free software" libraries in Nim, that would be even better. :P
23:17:08libman*WHO write
23:20:12cheatfatelibman, python is not copyfree?
23:20:31libmanhttp://copyfree.org/standard/rejected :(
23:21:12libmanhttp://libman.org/img/bak/pythonSadLicenseFoo.png
23:24:22*nsf joined #nim
23:24:41*StarBrilliant quit (Ping timeout: 276 seconds)
23:24:54libmanPyPy and some fraction of packages are copyfree, but not the standard library isn't.
23:25:48cheatfatetoo many text for me about copyfree... is there `one sentence` explanation?
23:28:58libmanIt's a "line in the sand" against restrictive licenses; only "don't hurt me, don't pretend you wrote it" licenses like MIT make the cut; for people who want to liberate the free software industry from lawyers and socialists claiming that only Mommy Government's copyleft enforcement had made free software possible.
23:31:24*StarBrilliant joined #nim
23:31:27cheatfatelibman, i think you know that free software industry is a first step to socialism and socialism its just a light version of communism
23:31:33*arnetheduck quit (Ping timeout: 240 seconds)
23:34:15libmanNope, free software is a natural and inevitable free market capitalist phenomenon. People do it for their rational self-interest.
23:52:39cheatfateI dont understand Microsoft... they introduced in Windows Vista GetQueuedCompletionStatusEx() function, but this function dont give any significant speed increase vs old GetQueuedCompletionStatus()
23:58:56cheatfateits like 0.01 ms