<< 19-11-2016 >>
00:14:32*dmi0 joined #nim
00:15:56*yglukhov quit (Remote host closed the connection)
00:16:49*nsf quit (Quit: WeeChat 1.6)
00:25:06*dmi0 quit (Ping timeout: 250 seconds)
00:29:06*vlad1777d quit (Remote host closed the connection)
00:31:39*yglukhov joined #nim
00:36:04*yglukhov quit (Ping timeout: 258 seconds)
00:37:51*djiin joined #nim
00:38:03*Jesin quit (Quit: Leaving)
00:40:20*Jesin joined #nim
00:41:08*saml_ joined #nim
00:41:22*Matthias247 quit (Read error: Connection reset by peer)
00:56:18*pregressive quit (Remote host closed the connection)
00:56:55*pregressive joined #nim
01:01:23*pregressive quit (Ping timeout: 252 seconds)
01:05:56*pregressive joined #nim
01:07:44*pregressive quit (Remote host closed the connection)
01:08:07*pregressive joined #nim
01:09:13*pregressive quit (Remote host closed the connection)
01:09:49*pregressive joined #nim
01:14:25*yglukhov joined #nim
01:14:30*pregressive quit (Ping timeout: 250 seconds)
01:18:48*yglukhov quit (Ping timeout: 245 seconds)
01:32:55*dmi0 joined #nim
01:39:16FromGitter<GraydenH> in the net package why does acceptAddr need a new socket
01:40:02FromGitter<GraydenH> seems like it would be more normal to keep using the socket you used to call acceptAddr
01:40:40FromGitter<GraydenH> also how is two way communication usually done?
01:41:05FromGitter<GraydenH> I can't seem to get them to talk back and forth
01:41:24FromGitter<GraydenH> client.nim:
01:41:43FromGitter<GraydenH> import net ⏎ ⏎ ```code paste, see link``` [https://gitter.im/nim-lang/Nim?at=582fadd7613368f619cede67]
01:41:53FromGitter<GraydenH> server.nim:
01:42:05FromGitter<GraydenH> import net, strutils ⏎ ⏎ var socket = newSocket() ⏎ socket.bindAddr(Port(1234)) ⏎ socket.listen() ... [https://gitter.im/nim-lang/Nim?at=582faded5eb8ea792a3a8afd]
01:42:19FromGitter<GraydenH> import net, strutils ⏎ ⏎ ```code paste, see link``` [https://gitter.im/nim-lang/Nim?at=582fadfb2cf343a318c4472f]
01:45:29*brson quit (Quit: leaving)
01:56:43*yglukhov joined #nim
01:59:30*arnetheduck joined #nim
02:00:59*yglukhov quit (Ping timeout: 244 seconds)
02:33:46FromGitter<cheatfate> @GraydenH first of all you can't use socket your used to call acceptAddr, just because accept(3) function creates connected socket by itself.
02:37:29*yglukhov joined #nim
02:41:46*yglukhov quit (Ping timeout: 250 seconds)
02:47:14*chemist69 quit (Ping timeout: 250 seconds)
03:01:01*chemist69 joined #nim
03:19:56*yglukhov joined #nim
03:24:30*yglukhov quit (Ping timeout: 250 seconds)
03:42:49*zaquest joined #nim
04:02:16*yglukhov joined #nim
04:06:55*yglukhov quit (Ping timeout: 268 seconds)
04:12:01FromGitter<GraydenH> o k
04:20:30*saml_ quit (Remote host closed the connection)
04:28:22FromGitter<GraydenH> server:
04:28:24FromGitter<GraydenH> import net, strutils ⏎ ⏎ ```code paste, see link``` [https://gitter.im/nim-lang/Nim?at=582fd4e7b563b5516c3a51da]
04:28:45FromGitter<GraydenH> client:
04:28:47FromGitter<GraydenH> import net ⏎ ⏎ ```code paste, see link``` [https://gitter.im/nim-lang/Nim?at=582fd4fe5eb8ea792a3b1807]
04:30:08FromGitter<GraydenH> the server received the hello and prints it, then the client errors on the accept call
04:30:52FromGitter<GraydenH> client.nim(17) client ⏎ net.nim(679) accept ⏎ net.nim(597) acceptAddr ⏎ os.nim(151) raiseOSError ⏎ Error: unhandled exception: An invalid argument was supplied. ... [https://gitter.im/nim-lang/Nim?at=582fd57c238757566cc5a86e]
04:34:18FromGitter<GraydenH> so I'm guessing client is the invalid parameter
04:44:41*yglukhov joined #nim
04:46:37*dddddd quit (Remote host closed the connection)
04:48:13*chemist69 quit (Ping timeout: 256 seconds)
04:49:08*yglukhov quit (Ping timeout: 260 seconds)
05:15:16*chemist69 joined #nim
05:19:20*dmi0 quit (Ping timeout: 250 seconds)
05:20:05*space-wizard quit (Quit: My MacBook has gone to sleep. ZZZzzz…)
05:48:12*yglukhov joined #nim
05:52:17*yglukhov quit (Ping timeout: 240 seconds)
06:00:30FromGitter<cheatfate> @GraydenH looks like you not properly understand how bsd sockets working.
06:01:23FromGitter<cheatfate> @GraydenH i think you need to read some manuals about tcp/udp socket functions first
06:07:35FromGitter<GraydenH> I think I know enough...
06:09:24FromGitter<GraydenH> every example I've found does the same thing for the server I have
06:10:11FromGitter<GraydenH> simply sends something back to the socket it gets from accept
06:10:44FromGitter<GraydenH> I just can't figure out what the client side is supposed to look like
06:11:03FromGitter<GraydenH> no need to be so patronizing.
06:11:35FromGitter<endragor> @GraydenH only server accepts. with client you just connect and send/recv stuff right away
06:14:00FromGitter<endragor> your server-side code is also wrong in a couple of places. The examples you've found must be different, take a closer look
06:27:14*jh32 joined #nim
06:27:45FromGitter<GraydenH> client: ⏎ ⏎ ```code paste, see link``` [https://gitter.im/nim-lang/Nim?at=582ff0e02cf343a318c548a1]
06:28:37FromGitter<GraydenH> what is wrong with the server?
06:29:10FromGitter<GraydenH> is the asyncnet api different? because most of the examples use that
06:29:21FromGitter<GraydenH> https://rosettacode.org/wiki/Echo_server#Nim
06:29:29FromGitter<GraydenH> http://goran.krampe.se/2014/10/25/nim-socketserver/
06:29:45FromGitter<GraydenH> http://forum.nim-lang.org/t/1769
06:30:33*yglukhov joined #nim
06:35:08*yglukhov quit (Ping timeout: 260 seconds)
06:35:42FromGitter<GraydenH> ah, recvline needs a newline
06:37:41FromGitter<GraydenH> got it working
06:38:03FromGitter<GraydenH> still not sure why the server is incorrect
06:46:58FromGitter<endragor> what were these programs supposed to do? ⏎ ⏎ 1) there is `while true` loop on client side, but server responds to the same client just once, then accepts a new one ⏎ 2) server doesn't close client sockets, so you'll run out of file descriptors there if many clients connect ⏎ ... [https://gitter.im/nim-lang/Nim?at=582ff562e712c9a1188650c7]
06:51:44*chemist69 quit (Ping timeout: 258 seconds)
06:54:44*chemist69 joined #nim
06:57:12*yglukhov joined #nim
07:01:57*yglukhov quit (Ping timeout: 256 seconds)
07:11:22FromGitter<GraydenH> not learning networking, just had trouble figuring out the exact functions to call for this module
07:12:17FromGitter<GraydenH> the logic design comes after it actually works
07:12:31*nsf joined #nim
07:39:36*yglukhov joined #nim
07:44:01*yglukhov quit (Ping timeout: 260 seconds)
08:05:52*Demos joined #nim
08:11:48*yglukhov joined #nim
08:12:50*Demos_ joined #nim
08:15:33*Demos quit (Ping timeout: 248 seconds)
08:16:38*yglukhov quit (Ping timeout: 256 seconds)
08:38:07*yglukhov joined #nim
08:38:52*zevlg quit (Read error: Connection reset by peer)
08:42:48*LeNsTR quit (Changing host)
08:42:48*LeNsTR joined #nim
08:45:58*yglukhov quit (Ping timeout: 252 seconds)
08:50:23*Matthias247 joined #nim
08:54:39*Arrrr joined #nim
08:58:35FromGitter<GraydenH> import net, os ⏎ ⏎ ```code paste, see link``` [https://gitter.im/nim-lang/Nim?at=5830143bb563b5516c3b4b5f]
08:59:08FromGitter<GraydenH> why does this not properly send to all the client sockets in the seq
08:59:32FromGitter<GraydenH> but switching between them they all work
09:00:26FromGitter<GraydenH> just iterating through args.client doesn't work
09:00:41FromGitter<GraydenH> args.clients I mean
09:06:44*gokr joined #nim
09:08:25Araqdon't use threads if you have no idea how they work please
09:09:11Araqthread creation in a while loop is wrong as is reusing the thread variable's memory for new threads
09:11:20*yglukhov joined #nim
09:17:40*yglukhov quit (Ping timeout: 250 seconds)
09:19:36FromGitter<GraydenH> how else would a server accept an indeterminable amount of connections
09:20:08FromGitter<GraydenH> you guys have a bad habit of telling people they're wrong without explaining why
09:21:57*gokr quit (Ping timeout: 240 seconds)
09:22:02*yglukhov joined #nim
09:23:32*Salewski joined #nim
09:23:56ArrrrIs easier
09:25:31SalewskiGraydenH, was the twitter clone chapter of Dom's book not helpful for "server accept" stuff?
09:26:05FromGitter<GraydenH> have not been reading it
09:26:11SalewskiI read it some months ago, was interesting foe me, but I am not really interested in web development currently.
09:26:35FromGitter<GraydenH> I think I have the sockets figured out though
09:26:53FromGitter<GraydenH> just figuring out threads now
09:28:11Araqwhy not start with 'spawn' then? gokr has a blog post of how to do a server with spawn in like 10 lines of code
09:28:28Araqor use async
09:29:04Araqnobody starts thread programming with 'createThread', it's the low level primitive full of gotchas
09:30:13Araqyou don't even joinThread()
09:31:34FromGitter<GraydenH> I attempted the most familiar version
09:32:11FromGitter<GraydenH> http://goran.krampe.se/2014/10/25/nim-socketserver/
09:32:17FromGitter<GraydenH> that blog?
09:33:14Araqyes
09:34:28FromGitter<GraydenH> having three different concurrency models can be confusing
09:37:56Araqblame the OS for supporting C apps with their huge linear stack spaces
09:38:15Araqrequiring a different model for IO bound concurrency (async)
09:42:19AraqC++, Java, C# all have threads, thread pools and async frameworks.
09:43:51FromGitter<GraydenH> not in the standard library
09:47:02Araqlol
09:47:23Araqsurely they have this in their std libs, maybe C++ doesn't
09:48:05flyxC++11 has it.
09:48:13*yglukhov quit (Remote host closed the connection)
09:49:52FromGitter<GraydenH> not async
09:50:04*Salewski left #nim (#nim)
09:50:08FromGitter<GraydenH> c# is the only one with async
09:50:42flyxhttp://en.cppreference.com/w/cpp/thread/async
09:51:17FromGitter<GraydenH> oh
09:51:33FromGitter<GraydenH> interesting
09:51:38ArrrrJava is the one without async support
09:51:47FromGitter<GraydenH> yeah
09:52:10ArrrrBut it has a lot of good stuff https://docs.oracle.com/javase/8/docs/api/?java/util/concurrent/package-summary.html
09:52:35flyxdepends on whether you count JavaEE as stdlib
09:53:41ArrrrNo no, that's standard
09:53:47ArrrrOut of the box
09:54:18FromGitter<GraydenH> `expression 'register(selector, getFd(server), {EvRead}, nil)' has no type (or is ambiguous)`
09:54:38FromGitter<GraydenH> I can't see an override that makes that ambiguous
09:54:46FromGitter<GraydenH> http://nim-lang.org/docs/selectors.html
09:57:15*bjz joined #nim
10:08:41*yglukhov joined #nim
10:10:06*yglukhov quit (Remote host closed the connection)
10:13:42*yglukhov joined #nim
10:20:47*Demos joined #nim
10:21:29FromGitter<GraydenH> yeah can't figure out this error
10:21:56FromGitter<endragor> @GraydenH it usually appears when you try to assign `void` return value to something
10:21:57*vlad1777d joined #nim
10:22:57FromGitter<endragor> @Araq what are your thoughts on turning the stdlib's Queue into full Deque? At the moment there are procs to peek first/last values, but no `removeLast`/`addFirst`. Not sure about future naming though, it already feels a bit messy there. One way to go is to deprecate current names in favor of addFirst/removeFirst/peekFirst and addLast/removeLast/peekLast, and rename Queue to Deque.
10:23:48*Demos_ quit (Ping timeout: 260 seconds)
10:26:17*Demos quit (Ping timeout: 240 seconds)
10:29:38FromGitter<endragor> @GraydenH btw you don't need selectors for async code. see https://rosettacode.org/wiki/Echo_server#Nim
10:30:56*djiin quit (Ping timeout: 244 seconds)
10:31:22*vlad1777d quit (Read error: Connection reset by peer)
10:33:02Araqendragor: I have no opinion on Queue but lists.nim should be deprected in favour of 4-5 templates
10:33:22*vlad1777d joined #nim
10:34:00FromGitter<endragor> 4-5 templates?
10:35:27FromGitter<GraydenH> thats not spawn
10:36:15Araqsomething like template listAdd(head, node: untyped; next=next) = if head.isNil: head = node else node.next = head; head = node
10:36:50Araqso that's it's actually useful for when I do use lists ...
10:37:05FromGitter<GraydenH> how is the selectorData used
10:41:25*vlad1777d quit (Remote host closed the connection)
10:41:46FromGitter<endragor> @GraydenH how do you plan to use spawn with selectors? async IO is based on selectors, just use it, it's the easiest (and efficient) way to implement the chat-like thing you were trying to implement.
10:49:31FromGitter<Almynic> I have a silly question : Are includes guarded in nim? In C which I know, you have to guard includes manually.
10:57:02FromGitter<endragor> in Nim you shouldn't use includes. `import` is the way to import modules. ⏎ C doesn't really have a notion of module on language level, so you have to use header files and includes and include guards to "simulate" modules.
10:59:40FromGitter<Almynic> I'm not including a module i'm include another nim file
11:01:50FromGitter<endragor> then if you think it could ever end up being included twice in the same module, you're doing something wrong
11:03:31FromGitter<Almynic> Tested it now if I include the same thing twice I get a redefinition error on compilation as it should be. Thanks for your response
11:04:58FromGitter<GraydenH> what happened
11:05:49FromGitter<GraydenH> how would i reply to all connections with that async pattern
11:06:12FromGitter<GraydenH> for chat like functionality like you said
11:15:52FromGitter<endragor> @GraydenH quite like you tried to do it with threads
11:17:29dom96hi guys
11:17:33FromGitter<endragor> @GraydenH https://gist.github.com/endragor/1528924d9ae01fab5df5f1e6f5ea8746
11:17:48FromGitter<endragor> something like that should work (didn't test)
11:20:27FromGitter<endragor> and it's not entirely correct with the current stdlib with regards to send(), because current stdlib doesn't handle multiple pending sends to the same socket at the same time, but that will do as a toy example
11:22:30FromGitter<endragor> @dom96 hi! could you take a look at my message above wrt Queue/Deque? What do you think about adding `deques` module and perhaps deprecating `queues` (because Queue API is a subset of Deque).
11:28:20dom96endragor: Sounds good. I don't like that the queues module defines a pop proc, perhaps I'm wrong but isn't that normally a stack operation?
11:38:06dom96endragor: GraydenH: My book literally has a chat app example, have you guys seen it? https://github.com/dom96/nim-in-action-code/tree/master/Chapter3/ChatApp
11:39:11dom96It uses threads only for reading from the stdin stream.
11:39:50dom96The code in that repo has many comments which explain it in great detail, plus my book explains it in even more detail :)
11:41:50*dmi0 joined #nim
11:44:36AraqEndragor: we also need proper priority queues and cache data structures
12:02:18*Snircle joined #nim
12:09:34FromGitter<cheatfate> @endragor
12:09:37FromGitter<cheatfate> and it's not entirely correct with the current stdlib with regards to send(), because current stdlib doesn't handle multiple pending sends to the same socket at the same time
12:09:49FromGitter<cheatfate> this is not stdlib problem
12:10:04FromGitter<cheatfate> this is OS problem,
12:10:42FromGitter<cheatfate> try to make many concurrent calls to send on non-blocking socket and you will see...
12:33:37cheatfateAraq, is it possible to make this https://github.com/nim-lang/Nim/issues/4332 while you are working on codegen?
12:34:44*Demos joined #nim
12:35:50Araqah yeah, that's exactly what my sighashes needs anyway
12:36:03dom96cool, Gitlab now sends emails when pipelines fail
12:36:25dom96It will serve as a good reminder to fix the windows tests :)
12:37:10cheatfateAraq, and maybe this will be also not so hard to fix? https://github.com/nim-lang/Nim/issues/4763
12:37:36cheatfateor at least mark it with `codegen` label
12:37:53*chemist69 quit (Ping timeout: 258 seconds)
12:37:56dom96cheatfate: Brilliant idea :)
12:39:01*chemist69 joined #nim
12:39:30AraqI'd rather remove .packed. I'm tired of this C bullshit. If you know better than your compiler than fucking do the size and offset computations by hand
12:40:46cheatfateAraq, if you remove {.packed.}, then we will need to introduce more python's bullshit like struct.pack()/struct.unpack()
12:41:20cheatfatebecause this {.packed.} mostly needed for fast processing binary network data
12:41:35cheatfatebecause network data is not aligned
12:41:51cheatfatebinary network data/binary file data etc
12:42:26Araqstruct.pack/unpack are nicer to work with anyway since you can specify endianess
12:42:43cheatfateso please, do not remove {.packed.}
12:42:48Araqplus we don't need fucking strings to describe the layout, we have DSL support for this
12:42:55Araqmacros
12:43:40Araqcheatfate: let's say I won't remove it before a nicer package to deal with the problem becomes available
12:44:50cheatfatei dont think something will be nicer then, cast[SomePacketDataPtr](memoryBuffer)
12:45:32Araqthat doesn't work for big endianness right now, so surely it will be "nicer".
12:45:44Araqit will actually be a solution, not a hack.
12:48:14cheatfateAraq, is it so hard to fix this {.pragma.} placement in codegen? why you are so opposite?
12:48:54Araqbecause it's unlikely to work with the JS or LLVM backends
12:48:56DemosI mean for network stuff you need to convert everything to host byte order anyway
12:49:17Araqand you are not in control, you are depending on the C compiler.
12:49:37Araq.packed is about control and Nim's way doesn't control anything
12:49:58AraqI'm opposite because it's the wrong solution.
12:50:34cheatfateDemos, this depends on protocol, not on network...
12:50:49cheatfateAraq, there no problems with LLVM backend
12:51:54cheatfateDemos, and what you can say about binary files?
12:52:31cheatfateAraq, and i dont think JS handle this pragma in current implementation
12:52:51*elrood joined #nim
12:54:47cheatfateAraq, i have tested this pragma with LLVM on FreeBSD/clang
12:54:56cheatfateand it works
12:55:18cheatfatejavascript can't support it because it stupid interpreter
12:58:46Demosyeah for binary I think it's good
12:58:51DemosI don't have a problem with the pragma
12:59:08Demosalso almost all network protocols are big endian
13:02:33cheatfateAraq, `System programming features: Ability to manage your own memory and access the hardware directly. Pointers to garbage collected memory are distinguished from pointers to manually managed memory.`
13:04:12cheatfateJS backend could not satisfy this Nim's feature, but c backend can, so why you trying to remove stable and very useful {.packed.} in favor of Javascript?
13:07:14*brechtm joined #nim
13:08:01Araqit is not stable, you reported a bug about it
13:09:24Araqbbl
13:13:32*vlad1777d joined #nim
13:20:39cheatfateAraq, its not stable because for some reason you are adding `#pragma pack(pop)` after structure declaration
13:20:47cheatfateyou are not adding
13:20:48cheatfate*
13:21:11*vlad1777d quit (Remote host closed the connection)
13:39:17*brechtm quit (Remote host closed the connection)
13:59:49*desophos quit (Read error: Connection reset by peer)
14:00:17*dddddd joined #nim
14:09:02*gokr joined #nim
14:22:32*dmi0 quit (Ping timeout: 258 seconds)
14:25:20*bjz_ joined #nim
14:26:06*bjz quit (Ping timeout: 256 seconds)
14:27:14*gokr quit (Ping timeout: 256 seconds)
14:30:40*yglukhov quit (Remote host closed the connection)
14:39:51*aziz joined #nim
15:01:03*bjz joined #nim
15:01:40*bjz_ quit (Ping timeout: 244 seconds)
15:07:55Demoshave yall played around with GCC's concepts support (for c++)
15:08:07Demosit feels nimish!
15:08:58ArrrrIs it better or worse than nim's ?
15:10:41Demoswell it's more stable
15:10:48Demosand has better defined semantics
15:11:00Demoslast I checked nim's typeclasses are pretty experiemntal
15:11:07Demosbut the implicit template syntax is the same
15:11:18Demosexcept without the shortcuts to access the type
15:11:46Arrrrin nim i can write proc test[T: not (ref | string)]
15:11:51ArrrrIs it possible to do that?
15:11:52Demoslike in nim
15:11:52Demosproc foo(a) = return a.T(...)
15:12:00Demosyeah
15:12:04DemosI think so
15:12:18DemosI mean that's already technically possible in c++
15:12:31Demosbut it's not pretty
15:12:41Demosconcepts should make it pretty
15:12:47Demosand make a bunch of other stuff pretty
15:12:50Demoslike being able to say
15:13:07Demosauto foo(auto a, auto b) { return a+b; }
15:13:34Demosand have it be expanded to template<class R, A, B> R foo(A a, B b) { return a + b; }
15:13:37Demoswhich nim can already do
15:14:51ArrrrWell, zahary was about to dope concepts http://forum.nim-lang.org/t/2396#14817
15:14:59ArrrrI don't know how the thing is going on
15:16:19Demoswould concept[T] be even higher kinded types?
15:17:40ArrrrI think that was the idea, read the full thread
15:18:22*saml_ joined #nim
15:21:12DemosI ended up back in c++ land because intertia, so now I'm pining for modern features
15:21:50Demosbut hey modules is a TS now and so we should se implementations pretty soon
15:22:16Demoslike 15 years too late .....
15:31:07*yglukhov joined #nim
15:35:32*yglukhov quit (Ping timeout: 260 seconds)
15:36:50*kulelu88 joined #nim
16:28:52*enthus1ast joined #nim
16:34:39enthus1ast i've written code to parse nmap style ip range syntax, maybe someone find it useful: https://github.com/enthus1ast/nimIpAddresses
16:37:50*Dankrad joined #nim
16:38:55DankradIs it possible to get the time in nimscript?
16:39:16DankradAnd also the date
16:41:20federico3a quick way to "map" a JsonNode into a regular object would be useful
16:46:36federico3to[MyObject](raw_json_string) works :)
16:48:04*Demos_ joined #nim
16:50:40*Demos quit (Ping timeout: 260 seconds)
16:53:55*Snircle quit (Quit: Textual IRC Client: www.textualapp.com)
16:58:38*yglukhov joined #nim
17:04:56*gangstacat quit (Quit: Ĝis)
17:06:50dom96Dankrad: staticExec("date") is the only way I'm afraid.
17:06:58dom96federico3: I wouldn't rely on that if I were you
17:07:08dom96The format that marshal uses may change
17:07:14federico3uh?
17:07:16federico3ouch
17:09:28*gangstacat joined #nim
17:30:59FromGitter<endragor> @cheatfate hey! I was away when you replied regarding send queueing. stdlib of a language and OS are different things. library can ease some burden of the user. that chat sample is one example when you need to manually ensure no interference between sends, and there are many more. the current design makes it easy to create a subtle error in networking code. I don't see why sends couldn't be queued on the lib level.
17:34:55FromGitter<endragor> actually, @dom96 book sample seems to have this subtle error
17:35:22dom96what error is this?
17:36:04*gokr joined #nim
17:36:41FromGitter<endragor> it's possible that two sends to the same socket will get to asyncdispatch at the same time, and asyncdispatch doesn't handle that case properly.
17:36:51*libman joined #nim
17:36:54FromGitter<endragor> we discussed that some time before
17:37:56dom96Isn't that only a concern when send()ing from multiple threads?
17:38:26cheatfateendragor: its not possible with `await`, but possible with `asyncCheck` and only first asyncCheck will be successful all other asyncCheck's will fail with EAGAIN
17:38:37*djiin joined #nim
17:43:46cheatfateendragor: hmm, but i think you are right, `asyncCheck` can cause problems because asyncdispatch not performs real call to send(), but registers write callback. And because there sequence of callbacks for corresponding socketFd, its possible to register any number of `send` operation and `poll` will resolve them all after first `send` will be completed.
17:44:51*Demos joined #nim
17:45:08cheatfateso after this only one `send` will be really completed, and all other registered send operations will be `not really` completed.
17:45:36*dom96 still has no idea what the problem is
17:46:11dom96Is the problem that sends will not be sent in the order that they were executed with `asyncCheck`?
17:46:14*eizua joined #nim
17:46:19cheatfatedom96, nope
17:47:01cheatfatedom96, take a look on https://github.com/nim-lang/Nim/blob/devel/lib/pure/asyncdispatch.nim#L1191
17:47:14*Demos_ quit (Ping timeout: 258 seconds)
17:48:42dom96I think I see the problem, is it partial send()s?
17:48:52cheatfatedom96, nope
17:49:08cheatfateif we make many calls to asyncCheck send()
17:49:08dom96Then what?
17:49:22FromGitter<endragor> as I remember: ⏎ ⏎ 1) the upcoming asyncdispatch will just silently overwrite existing write callback if new write callback is registered. ⏎ 2) the current asyncdispatch goes through all registered callbacks if Write event is triggered, so "future" sends may interfere [https://gitter.im/nim-lang/Nim?at=583090a2df9f0f6e7f576e43]
17:50:21*krux02 joined #nim
17:50:35cheatfateok, for example user made many calls like this `asyncCheck send(sock, ....)`
17:50:48dom96endragor: yes, so the way I see it is that the first `send` doesn't finish sending everything.
17:50:55dom96There are other `send`s queued
17:51:09dom96the next time a Write event occurs a different send is executed.
17:51:33dom96and the rest of the data from the first `send` is sent at some further point in the "future" and you get corrupted data
17:51:34dom96Right?
17:51:35cheatfatedom96, all other `sends` will be resolved
17:51:45cheatfatewhen processing first `write` event
17:52:38cheatfatethere will be no one by one operation as you described
17:53:06cheatfateall pending callbacks will be called after just one incoming `write` event
17:53:13dom96okay, so if a `send` isn't able to complete
17:53:22dom96the rest of the callbacks shouldn't be "resolved"
17:53:41cheatfatethere 2 ways
17:54:08cheatfate1. enable commented line https://github.com/nim-lang/Nim/blob/devel/lib/pure/asyncdispatch.nim#L1219
17:55:31cheatfate2. implement sending queue for asyncdispatch...
17:55:49*irrequietus joined #nim
17:56:07cheatfatesending queue will be moderate task, and i dont think it really needed
17:56:33FromGitter<endragor> that line doesn't seem correct to me - it allows new send to be executed even though there may be an existing send
17:57:12dom96we already have a queue though
17:57:17dom96the callbacks are a queue
17:57:19cheatfateendragor: this line absolutely correct (and i dont know reason why it was commented).
17:57:34dom96The comment specifies why
17:57:43dom96I only added it as an optimisation
17:58:07cheatfatedom96, it must not cause any crashes
17:58:18dom96it did when I was writing this code
17:58:27dom96but perhaps it was something else
17:58:31dom96in either case
17:58:37dom96I don't see how this solves any problem
17:58:45FromGitter<endragor> @cheatfate why is it correct? say, I call send() two times (the case we are investigating here), then the second send syscall will be invoked, even if the first one may not have been fully completed.
17:59:24*Snircle joined #nim
17:59:37cheatfateendragor: yeah, and it will be finished with EAGAIN error (which indicate that there pending send operation)
18:00:21cheatfateso it indicate that you are making something wrong in your code
18:00:44cheatfatelike trying to perform many `asyncCheck send(sameSocket, ...)`
18:01:15dom96The solution is to stop calling the rest of the callbacks after this line https://github.com/nim-lang/Nim/blob/devel/lib/pure/asyncdispatch.nim#L1075
18:02:04dom96The callback wants to be called again, which implies that it has not completed fully and so other callbacks should not be called.
18:02:20cheatfatedom96, its not very good solution because of this https://github.com/nim-lang/Nim/blob/devel/lib/pure/asyncdispatch.nim#L1197
18:02:37dom96because of what?
18:02:49cheatfatedom96, our `send` operation tries to send whole buffer
18:02:57FromGitter<endragor> I don't think there is a guarantee you'll get EAGAIN in that case. Imagine this scenario: ⏎ ⏎ 1) asyncCheck send(sameSocket) - sends some chunk. ⏎ 2) read(someOtherSocket) ⏎ 3) poll() returns read event ... [https://gitter.im/nim-lang/Nim?at=583093d1a5bc784f5657e556]
18:03:19cheatfatedom96, but it happens often when only part of buffer was sent via `posix.send`
18:03:20dom96cheatfate: what's the problem with that?
18:03:26dom96yes, exactly
18:04:02dom96so poll will be called again, and once an EvWrite event happens the `cb` will be called once more and the remaining data will be sent
18:04:43cheatfatedom96, yep
18:05:13dom96That's exactly what we want to happen so I don't see the problem
18:05:22cheatfateendragor: it will guarantee to avoid `asyncCheck send(sameSocket, ...); asyncCheck send(sameSocket, ...); asyncCheck send(sameSocket, ...)`
18:05:38*Demos_ joined #nim
18:06:25FromGitter<endragor> @cheatfate what will guarantee, again? You know that EAGAIN doesn't cause async operations to fail, right? They will just try again next time Write event occurs.
18:07:34cheatfateendragor: try to make sample code and make like 100 posix.sends to non-blocked socket
18:07:56*Demos quit (Ping timeout: 258 seconds)
18:08:13cheatfateendragor: if sending buffer will be more, then socket buffer size
18:08:48cheatfateok its hard to explain
18:10:23cheatfateEAGAIN happens when socket is non-blocked and you call send(3) and operation can't be completed immediately
18:10:51FromGitter<endragor> @cheatfate I know what would happen, but we're talking about Nim here. What I'm saying is: ⏎ ⏎ 1) If that line is uncommented, the undesired behaviour (interfering sends) is possible. ⏎ 2) I don't see why `asyncCheck send(socket)` is a bad thing app-design-wise. Besides, multiple sends to the same socket is possible even without that - see that chat app for example. [https://gitter.im/nim-lang/Nim?at=583095aa3418b2e57f2b7f23]
18:11:33dom96endragor: what do you think about my proposed solution?
18:12:17*StarBrilliant quit (Quit: ZNC - http://znc.in)
18:13:00FromGitter<endragor> @dom96 sounds good to me
18:14:35*Demos joined #nim
18:14:42cheatfatedom96, it will not resolve a problem...
18:15:19cheatfatejust because you can't identify what data was sent and what complete callback must be completed
18:16:06dom96cheatfate: yes, I can. The `send` proc keeps track of how much data was sent
18:16:13dom96And the callbacks are always in order
18:16:30cheatfateendragor: its a bad app-design-wise, because you can always lost your data when network buffer of socket will be overflowed
18:17:23FromGitter<endragor> @cheatfate how? do you remember we are talking about Nim's asyncdispatch.send() here? Because it feels to me you're talking about POSIX APIs.
18:17:31*Demos_ quit (Ping timeout: 258 seconds)
18:17:37FromGitter<endragor> Nim's send() doesn't complete before all data is sent
18:19:07cheatfatedom96, asyncCheck send(sameSocket, 1_000_000 bytes); asyncCheck send(sameSocket, 1 byte); asyncCheck send(sameSocket, 10 bytes)
18:19:32cheatfate3 callbacks registerd
18:21:06cheatfate1st callback will be completed successfuly
18:21:25cheatfate2nd callback cause EAGAIN
18:21:33cheatfate3rd callback cause EAGAIN
18:22:47FromGitter<Almynic> Why does this not work? I get this error ` could not import: SHA256 `on runtime ⏎ ⏎ ```code paste, see link``` [https://gitter.im/nim-lang/Nim?at=58309877613368f619d2b7db]
18:23:24*enthus1ast quit (Ping timeout: 250 seconds)
18:24:06*enthus1ast joined #nim
18:24:58cheatfateAlmynic: you miss name of imported procedure, try to look at https://github.com/nim-lang/Nim/blob/devel/lib/wrappers/openssl.nim
18:25:53FromGitter<endragor> @cheatfate 1) EAGAIN is not a problem. 2) if 1st callback is actually completed, 2nd may send successfully, too 3) With the suggested solution, 3rd callback will not be called if 2nd got EAGAIN.
18:26:17cheatfatewhy 3rd will not be called
18:27:03FromGitter<endragor> because the idea is to call callbacks in order. future callbacks will notbe called before the first in the queue is completed.
18:27:33*Varriount|Phone quit (Ping timeout: 260 seconds)
18:28:25cheatfatebut 2nd must not be called until 1st will send all of data, because then we have very undefined stream of data
18:28:49cheatfatedeveloper requests 1_000_000 bytes to send
18:28:57cheatfatein 1st request
18:28:58FromGitter<endragor> yes! that's exactly what we are solving here
18:29:13FromGitter<endragor> you said: ⏎ > 1st callback will be completed successfuly
18:29:24FromGitter<endragor> which means the first send was finished, so the second can proceed.
18:29:58FromGitter<endragor> the callback is "completed successfully" only when ALL data has been sent
18:29:58*eizua quit (Quit: Leaving)
18:31:59*vlad1777d joined #nim
18:32:29FromGitter<GraydenH> 2
18:35:44cheatfateendragor, thanks now i understand
18:37:23cheatfateendragor: but i think we need to remove seq[], because `seq` was bad idea, and now it becomes even worse
18:37:30*space-wizard joined #nim
18:38:03FromGitter<endragor> @cheatfate great! good we are on the same page now. what do you mean by `seq[]`?
18:38:59cheatfateendragor: we must make callback holder data.readCBs and data.writeCBs not sequences
18:39:58cheatfateso we can omit memory allocations
18:40:08cheatfatemaybe linked list will be better here
18:41:17krux02Araq: how is the plugin called that supports "included from" comments in vim? I would like to create an issue on nim-mode, to also support that comment in nim-mode for emacs
18:42:19dom96cheatfate: that should be an easy thing to do
18:42:23FromGitter<Almynic> I still don't get it which procedure do I have to import?
18:43:39*vlad1777d quit (Remote host closed the connection)
18:43:58dom96Almynic: I bet it's the uppecase letters, try renaming the proc to sha256
18:44:38dom96You should find the openssl header files and check there to see what the signature and name of the procedure is
18:44:48cheatfatedom96, last time i have replaced this seq's with linked lists and asynchttpserver starts to eating memory like a monster
18:45:10dom96cheatfate: then perhaps it's not a good idea to replace them with linked lists? :P
18:45:25FromGitter<endragor> not sure how linked lists would be better. you still have to allocate their nodes and they add fragmentation. in this case we just need to add to the end, remove from the beginning (Queue), and recreations of the seq each poll() can be avoided. ⏎ ⏎ also, multiple read callbacks are always an error (as it seems to me), so queue can be avoided there.
18:46:35dom96endragor: I think multiple read callbacks can happen as well
18:47:16FromGitter<endragor> @dom96 I couldn't imagine when
18:47:36dom96same situation as the send examples I guess
18:48:19dom96although I guess there is no use cases for 'asyncCheck read()'
18:48:33*simplyianm_ joined #nim
18:48:47FromGitter<endragor> in case with recv(), there is always just one place where you recv things from the socket. I can't think of a sane example where there are two or more.
18:49:13cheatfateendragor: it can happens
18:49:27FromGitter<endragor> @cheatfate example?
18:49:56dom96like I said, same example as the send one.
18:50:24dom96even if it won't happen in practice I would still be reluctant to make it unsupported
18:50:38ArrrrStupid question ... does nim reuses string for numbers? for example $5, does it create a new string every time?
18:50:48FromGitter<endragor> @dom96 I'm not sure how "same" it is :) With recv() you read data, process it, then read more, and so on. why would you request two reads at the same time?
18:50:49cheatfateArrrr, yep
18:51:44ArrrrYep what
18:51:48FromGitter<endragor> @dom96 not sure, in my opinion it's fine to throw exception right away if another read callback is already registered. it means the user did a mistake.
18:53:34cheatfateendragor: its hard to explain, what do you know about `gdb` working in asynchronous mode?
18:55:52cheatfateendragor: if we not add this for readCBs then we can't use something like `asyncCheck send(sameSocket, ...); asyncCheck recv(sameSocket, ...);`
18:56:27cheatfatefirst added first served
18:57:25FromGitter<endragor> the question is why would you ever need an `asyncCheck recv`. How could you ignore the data you've received?
18:57:52cheatfateendragor: asyncCheck was just example
18:58:24cheatfateyou can use `asyncCheck` technique to implement some kind of callback hell
18:58:31FromGitter<endragor> I didn't get the point. What I'm saying is that there is absolutely no application where you want two simultaneous reads on the same socket
18:58:32cheatfatelike future.callback =
18:59:54*Dankrad quit (Ping timeout: 246 seconds)
19:01:03cheatfateendragor: something like this https://gist.github.com/cheatfate/720ab1868398ac0769c5ac79dab74bf5
19:02:05*simplyianm_ quit ()
19:02:16*Varriount|Phone joined #nim
19:02:54FromGitter<endragor> I don't see two simultaneous recv() there. In async code that would simply be: ⏎ await send() ⏎ await recv()
19:05:07*Demos_ joined #nim
19:06:58*Demos quit (Ping timeout: 258 seconds)
19:07:37dom96Anyway, fixed https://github.com/nim-lang/Nim/commit/d847d350097a36a6008fbb7ce056f470725a29a7
19:08:51dom96hrm, I think another bug is possible here
19:09:09dom96IIRC callbacks can add other callbacks
19:09:28dom96Which would cause the order to be messed up
19:10:38cheatfatedom96, i think you need to revert this commit
19:11:21cheatfateits wrong because now it can add 2 not finished callbacks
19:11:29cheatfateits wrong because now it can add not finished callback twice
19:12:05cheatfateor my wrong sorry
19:14:38cheatfatedom96, you are right callbacks are adding other callbacks
19:14:59dom96yep, fixing that now
19:22:07dom96and fixed
19:24:43cheatfateheh its not a good fix dom, now we have 2 allocations
19:25:08cheatfateso make asyncdispatch even slower
19:25:11cheatfatemade
19:26:37dom96The important thing is that it fixes the problem. We can always optimise later.
19:30:51cheatfatedom96, heh, i think `later` is you favorite word... and your `later` never happens
19:31:57elroodinstead of complaining unconstructively you could do yourself a favour and offer a better solution
19:32:08dom96indeed.
19:32:16dom96I don't understand why you love conflict so much
19:32:18dom96I don't owe you anything
19:36:14cheatfateelrood, maybe i need more time to make a better solution, didn't you thought about it?
19:38:11*Dankrad joined #nim
19:39:02*Jesin quit (Quit: Leaving)
19:39:22elrooddon't take this the wrong way, but i neither have the time nor intention to argue or discuss this with you. maybe it would be better to reduce your criticism to a constructive technical level instead of getting personally impolite towards dom96 or anybody else, is what i am saying
19:39:57*MrAxilus[m] quit (Ping timeout: 240 seconds)
19:40:03*M-Quora quit (Ping timeout: 245 seconds)
19:40:09*jivank[m] quit (Ping timeout: 246 seconds)
19:40:19cheatfateelrood, if you dont have a time to discuss this why you ever entered this discussion?
19:40:19*Guest73656[m] quit (Ping timeout: 258 seconds)
19:40:33*TheManiac quit (Ping timeout: 260 seconds)
19:40:33*hohlerde quit (Ping timeout: 260 seconds)
19:40:33*zielmicha[m] quit (Ping timeout: 260 seconds)
19:40:33*prose[m] quit (Ping timeout: 260 seconds)
19:40:49*erwana[m] quit (Ping timeout: 260 seconds)
19:40:49*terry[m] quit (Ping timeout: 260 seconds)
19:43:39cheatfateadding `not well-thought` code to just fixing issue which are used very rarely, by adding 1 more allocation and affecting many other projects which are based on this code, is this empty criticism?
19:43:49*Jesin joined #nim
19:44:36*byte512 quit (Ping timeout: 265 seconds)
19:45:28*saml_ quit (Remote host closed the connection)
19:45:45dom96It's a fair criticism. But what you said after is an attack on me personally.
19:46:14dom96Would you say something like that to a colleague?
19:47:32*Dankrad quit (Ping timeout: 260 seconds)
19:48:03cheatfateof course i will say
19:48:29elroodsometimes your wording comes across as patronizing or borderline rude, and that remark about dom's later was completely off and uncalled for. i guess most here value your technical input, but please in your own interest try to keep it constructive and polite
19:50:13elroodnow let's forget about this and move on. no harm done, no hard feelings
19:50:45cheatfatedom96, is there any other ways to push you to start working on issues which has `later` status already?
19:52:26dom96Being polite would certainly encourage that.
19:55:31dom96But also convincing me of the importance of these issues.
19:57:25cheatfatei do not think i was very cruel, but yeah it was sarcasm, and maybe it was too hard for you, sorry
19:59:34Arrrr:)
19:59:59cheatfateelrood: dont waste your time on me any more, i'm ignoring you
20:01:57elroodthat's an.. interesting reaction
20:02:07elroodbut i guess i can live with that
20:05:50*brson joined #nim
20:08:47*Ven joined #nim
20:11:19cheatfatedom96, also addRead/addWrite needs modification to avoid unnecessary calls to selectors `if len(readCBs/writeCBs) > 0`
20:17:50cheatfateendragor: is windows implementation is affected to this issue too?
20:22:28*Ven quit (Read error: Connection reset by peer)
20:22:59*Ven joined #nim
20:36:33*Arrrr quit (Quit: WeeChat 1.5)
20:36:49*aziz quit (Remote host closed the connection)
20:59:03*Demos joined #nim
21:01:37*Demos_ quit (Ping timeout: 240 seconds)
21:04:29*space-wizard quit (Quit: My MacBook has gone to sleep. ZZZzzz…)
21:04:37*brson quit (Quit: leaving)
21:05:14*space-wizard joined #nim
21:07:27*Demos_ joined #nim
21:09:58*Demos quit (Ping timeout: 268 seconds)
21:16:18*Sembei quit (Ping timeout: 250 seconds)
21:17:05*nsf quit (Quit: WeeChat 1.6)
21:19:26*dmi0 joined #nim
21:30:30*brson joined #nim
21:31:35*Demos joined #nim
21:34:10*Demos_ quit (Ping timeout: 258 seconds)
21:51:50*Demos_ joined #nim
21:52:54*libman quit (Quit: Leaving.)
21:53:20*Demos quit (Ping timeout: 258 seconds)
21:53:25*Ven quit (Read error: Connection reset by peer)
21:53:51*Ven joined #nim
22:09:38*Ven quit (Read error: Connection reset by peer)
22:09:52*Ven joined #nim
22:35:12*space-wizard quit (Quit: My MacBook has gone to sleep. ZZZzzz…)
22:35:56*Ven quit (Quit: My MacBook has gone to sleep. ZZZzzz…)
22:39:55cheatfateendragor: i have tried to implement test for this issue and failed... old asyncdispatch is working good on sequence of asyncCheck send() too
22:46:41*Demos joined #nim
22:49:18*Demos_ quit (Ping timeout: 258 seconds)
22:54:35*couven92 joined #nim
22:55:18*Demos quit (Read error: Connection reset by peer)
22:57:35*Demos joined #nim
23:12:49*gmpreussner quit (Quit: kthxbye)
23:13:02*space-wizard joined #nim
23:16:34*Demos_ joined #nim
23:17:57*chemist69 quit (Ping timeout: 240 seconds)
23:18:00*gmpreussner joined #nim
23:18:46*Demos quit (Ping timeout: 250 seconds)
23:18:54*chemist69 joined #nim
23:20:20*irrequietus quit (Ping timeout: 260 seconds)
23:23:05*couven92 quit (Read error: Connection reset by peer)
23:23:23*gmpreussner quit (Quit: kthxbye)
23:27:24*gmpreussner joined #nim
23:30:46*Demos joined #nim
23:31:28*Demos_ quit (Ping timeout: 258 seconds)
23:37:35*enthus1ast quit (Ping timeout: 256 seconds)
23:40:41*Demos quit (Read error: Connection reset by peer)
23:45:49*gokr quit (Read error: Connection reset by peer)
23:45:59*gokr joined #nim
23:48:14*gokr left #nim (#nim)
23:57:36*kulelu88 quit (Quit: Leaving)