<< 17-07-2019 >>

---

00:02:16	skrylar[m]	I dunno, CF was good in its time
00:02:32	skrylar[m]	Used to see a lot of argument about dev and maintenence TCO's between early PHPs and old coldfusion users
00:03:02	skrylar[m]	Nowadays there is also Lucee which can run most of it, albeit not the current gen adobe stuff
00:07:36	shashlick	Nim cpp should be enough
00:14:19	*	altarrel_ quit (Quit: Leaving)
00:17:39	*	altarrel joined #nim
01:03:36	*	altarrel quit (Quit: Leaving)
01:20:24	*	altarrel joined #nim
01:22:00	*	altarrel_ joined #nim
01:40:55	*	johnjane[m] joined #nim
01:41:58	*	altarrel quit (Quit: Leaving)
01:54:43	*	UNIcodeX quit (Read error: Connection reset by peer)
02:06:56	*	altarrel_ quit (Quit: Leaving)
02:10:09	*	krux02_ quit (Remote host closed the connection)
02:48:56	*	leorize quit (Ping timeout: 260 seconds)
02:49:52	*	leorize joined #nim
02:51:34	*	dddddd quit (Remote host closed the connection)
03:12:44	*	lritter quit (Ping timeout: 272 seconds)
03:13:11	*	lritter joined #nim
03:36:46	*	fjellfras joined #nim
03:38:11	*	lritter quit (Quit: Leaving)
03:44:25	*	ftsf joined #nim
04:07:54	*	martinium joined #nim
04:12:36	*	nsf joined #nim
04:22:42	skrylar[m]	looked in to loading ktx on nim
04:22:49	skrylar[m]	seems like the tools that make ktx are spread out here and there
04:36:02	*	martinium quit (Quit: Textual IRC Client: www.textualapp.com)
05:01:04	*	stefanos82 joined #nim
05:01:42	*	absolutejam joined #nim
05:01:59	shashlick	when you add to a seq, does it deepCopy or just point to T
05:02:14	*	NimBot joined #nim
05:03:47	*	Jjp137 quit (Ping timeout: 245 seconds)
05:04:21	*	Jjp137 joined #nim
05:05:11	shashlick	nightlies have passed - https://travis-ci.org/nim-lang/nightlies cc @narimiran
05:14:08	*	absolutejam quit (Ping timeout: 258 seconds)
05:14:44	*	absolutejam joined #nim
05:31:43	*	actuallybatman quit (Ping timeout: 245 seconds)
05:39:54	*	narimiran joined #nim
05:43:50	*	sniffdtek joined #nim
05:53:33	*	solitudesf joined #nim
06:16:03	*	solitudesf quit (Quit: Leaving)
06:16:30	*	solitudesf joined #nim
06:25:03	dvn	i can't find any docs for the tui module, but i saw someone using it. they did "import tui" - so it's part of the stdlib?
06:33:03	*	absolutejam quit (Ping timeout: 244 seconds)
06:41:36	*	fjellfras quit (Ping timeout: 268 seconds)
06:43:14	*	brakmic joined #nim
06:48:33	*	brakmic_ joined #nim
06:50:27	*	brakmic quit (Ping timeout: 245 seconds)
06:53:27	*	krux02 joined #nim
07:00:00	*	gmpreussner quit (Quit: kthxbye)
07:02:00	*	rockcavera is now known as Guest72744
07:02:00	*	tiorock joined #nim
07:02:01	*	Guest72744 quit (Killed (card.freenode.net (Nickname regained by services)))
07:02:01	*	tiorock is now known as rockcavera
07:04:46	*	gmpreussner joined #nim
07:06:58	*	absolutejam joined #nim
07:07:31	*	absolutejam1 joined #nim
07:08:27	*	tiorock joined #nim
07:11:19	*	brakmic_ quit ()
07:11:26	*	rockcavera quit (Ping timeout: 258 seconds)
07:11:52	*	brakmic joined #nim
07:14:12	*	brakmic quit (Read error: Connection reset by peer)
07:14:18	*	brakmic_ joined #nim
07:14:35	*	brakmic_ quit (Client Quit)
07:18:04	*	rockcavera joined #nim
07:21:01	*	tiorock quit (Ping timeout: 258 seconds)
07:22:49	*	jjido joined #nim
07:36:38	*	jjido quit (Quit: My MacBook has gone to sleep. ZZZzzz…)
07:37:50	Araq	dvn, I don't have tui.nim
07:38:13	narimiran	nim v0.20.2 is almost ready. if you're impatient, you can already grab it from https://github.com/nim-lang/nightlies/releases — please report ASAP if something is broken or not shipped :)
07:39:11	*	Jjp137 quit (Ping timeout: 244 seconds)
07:42:29	*	Jjp137 joined #nim
07:47:14	Zevv	wow 250 commits since 0.20.0
07:49:22	*	disruptek quit (Ping timeout: 248 seconds)
07:49:30	narimiran	Zevv: yeah, it's been only a month, but we worked hard on bugfixes :)
07:49:49	*	sniffdtek quit (Read error: No route to host)
07:51:45	*	disruptek joined #nim
07:53:44	FromGitter	<arnetheduck> fwiw, nim-chronos is gaining an asyncchannel for communicating work to the event loop (re flowvars and asyncevents etc)
08:00:13	FromGitter	<arnetheduck> and thinking about nim with batteries, I'd rather have a solid tool that allows me to package my deps myself so I get the ones I need (for the behind-the-firewall-case) rather than a bunch that I don't need, and missing some.. I still think that unless you think clearly about the "governance" of the deps list, it'll quickly become more annoying than the current setup - having a small std lib + tool is efficient
08:00:13	FromGitter	... because development can happen in parallel and independently, and because there's a natural tendency to let obsolete libraries die without breaking things - as long as the old versions are around, everything that worked keeps working and only when you want to upgrade or change things do you need to take into account ... [https://gitter.im/nim-lang/Nim?at=5d2ed58d570ac36f8d3c0e24]
08:01:04	Araq	well obviously "nim with batteries" is not for you, that doesn't mean the idea is bad
08:01:23	FromGitter	<arnetheduck> on governance, the crucial question is this: when do you drop a library whose maintainer has moved on? this is what plagues the std lib where someone randomly dumps some incomplete module and when it's time to deal with the tail 20% that make a quality implementation, there's silence and maintenance burden
08:02:35	FromGitter	<arnetheduck> well, it's totally for me if it's made in a way that I can supply my own list so I can include my own batteries as well
08:02:57	*	absolutejam quit (Ping timeout: 244 seconds)
08:03:35	Araq	I'm trying to see the newcomer's perspective. "oh a Nim that ships with a UI library, OpenGL support, SDL support and a better terminal.nim? Count me in."
08:04:36	Araq	that's different from "here, you can 'design' your own distro easily, but you still need to search Nimble and hope you picked the right one"
08:05:40	*	floppydh joined #nim
08:05:47	Araq	> on governance, the crucial question is this: when do you drop a library whose maintainer has moved on?
08:06:02	Araq	well ideally we don't, we fork the project and ensure it keeps compiling.
08:06:36	*	Jesin quit (Ping timeout: 272 seconds)
08:06:45	Araq	that's not good enough for a "battle tested Nim distro" but not every user has the same requirements or even quality standards
08:08:40	*	rokups joined #nim
08:09:52	Araq	take this issue as an example, https://github.com/nim-lang/Nim/issues/11713
08:10:25	Araq	yes, it's kinda embarrassing we never moved htmlparser into a Nimble package
08:11:00	Araq	but that aside, it's quite typical: it's not perfect at all but too useful to simple remove it from the face of the earth
08:12:41	Araq	and a nim-with-batteries should include it, a nim-solid shouldn't
08:23:04	dom96	arnetheduck: asyncchannel? can you elaborate on what this means?
08:23:19	*	ftsf quit (Quit: Leaving)
08:25:19	FromGitter	<arnetheduck> well - with limited time and resources, something's gotta give - which can most easily be outsourced to an interesting third party? htmlparser or in-depth researched language and compiler features? how do you encourage that it's light and easy for said newcomer to improve htmlparser? in a smaller library with clear bounds or in a large complicated setup? don't get me wrong - I agree with nim-batteries being a step
08:25:19	FromGitter	... in the right direction - for example, this way, there are pieces that are digestible on their own, without having to understand the full picture, which I think is helpful.
08:28:22	FromGitter	<arnetheduck> @dom96 we're developing a channel feature for chronos so we can pass work to it from other contexts / threads / etc.. @cheatfate and @zah will know more
08:29:48	dom96	so you're integrating Nim's channels with async?
08:30:17	dom96	We're doing the same but for spawn's FlowVar, and possibly might also do the same for channels if rayman22201 (or someone else) is interested
08:30:34	*	absolutejam joined #nim
08:31:52	federico3	arnetheduck: essentially you described distributions
08:32:23	Araq	federico3, yes it's about distributions
08:34:50	Araq	arnetheduck: well we agree htmlparser should be its own Nimble package
08:35:03	Araq	but now if we move it out of the stdlib, we break somebody's code
08:35:32	Araq	except if we had this nim-batteries thing, we could still ship it
08:35:41	*	absolutejam quit (Ping timeout: 268 seconds)
08:36:12	FromGitter	<mratsim> those kind of things need to be announced with 3months of leeway to manage people expectations
08:36:36	FromGitter	<mratsim> like "for 1.0 we will be moving some libraries out of the stdlib, this is done for .... here are the replacement"
08:36:46	FromGitter	<mratsim> just like basic2D and basic3d in the past
08:36:53	FromGitter	<arnetheduck> federico3, no, I mostly described go's vendor approach - distributions suffer the same problem: they package random versions of random packages which I may or may not be interested in (as in, my application may or may not be compatible with). look at modern distros - they're *also* moving away from the monolith - they package a stable core (glibc, gnome) and the rest goes into indiviudally upgradable modules with
08:36:53	FromGitter	... all deps included in the right versions (fedora atomic / modules / appstream)
08:37:48	Araq	mratsim: yes we did that in the past. we missed plenty.
08:37:50	FromGitter	<arnetheduck> Araq - better break someone's code before announcing a stable version than after, if it's going to be done - you don't break anyone's code if they stick with the previous nim version
08:38:03	federico3	the go model is a disaster
08:38:38	Araq	arnetheduck: well already people stick to their precious previous Nim versions, I don't think it's healthy
08:39:03	*	skaruts joined #nim
08:39:13	federico3	bundling is ultimately hurting end users (along with distributions on the way)
08:39:25	FromGitter	<mratsim> They don't stick to previous nim versions for something missing in the standard lib
08:39:43	FromGitter	<arnetheduck> well, when nim is smaller, it's easier to upgrade it..
08:40:06	Araq	they stick because Nim keeps breaking stuff and the dependencies work with 0.19.x
08:40:32	FromGitter	<arnetheduck> federico3, how is bundling hurting users?
08:40:57	FromGitter	<arnetheduck> there's less to break if nim is smaller
08:41:26	Araq	yeah, I like a smaller Nim as much as you do
08:41:46	FromGitter	<arnetheduck> if they need dependencies upgraded, and it has not been done yet (aka the dependency is not obsolete), you build in an incentive to get the dep upgraded as well
08:41:48	federico3	Araq: exactly. I have to use previous versions when I don't have time to port stuff to newer releases. At least depending exclusively on compiler + stdib allows me to use an older release without surprises.
08:42:31	FromGitter	<arnetheduck> https://docs.fedoraproject.org/en-US/modularity/ - what distros do nowadays
08:44:29	Araq	https://github.com/nim-lang/Nim/pull/11704 also *very* typical
08:44:41	Araq	it's a bugfix, the compiler is now correct.
08:44:42	federico3	arnetheduck: it discourage developers to care about backward compatibility and planned deprecation. This leads users to have large blobs of code where security upgrades are extremely difficult to do
08:44:58	federico3	(and also other upgrades, e.g. changes in network protocols)
08:44:59	Araq	but it breaks 2 Nimble packages
08:45:59	Araq	now, what to do?
08:46:22	Araq	the packages are declared to be "important"
08:46:38	Araq	(honest question, I don't know what to do :P )
08:47:07	federico3	Araq: in this case or in general?
08:47:54	FromGitter	<arnetheduck> developers are completely an utterly incabable of maintaining backwards compatibility at scale. glibc broke `memcpy` like 3 times - it's one of the most simple functions at the bastion of stability. seems like a bad idea to opimise your processes for that, no matter how much you'd like for the world to be both evolving and unchanging at the same time
08:48:46	FromGitter	<alehander42> Ь
08:48:46	Araq	federico3, in general, for this case I have a solution, but it will be more work
08:50:21	Araq	in general I'm of the opinion that v1 should be bug-compatible with what we ship. it's not the end of the world anyway, there will be a v2. it's like Nim never really understood how versioning works...
08:50:46	FromGitter	<arnetheduck> well, yeah - bugs spread and grow worse over time as people start relying on them. yet another reason to ensure that a std lib has a way to ship non-breaking upgrades.. in c++, that's inline namespaces whick allow the old and new implementations live side-by-side at an abi level - it's not perfect, but it's one approach. rust does a similar thing: deps of deps get versioned so that if a depents on c.2, and b depends
08:50:46	FromGitter	... on c.3, both c.2 and c.3 get bundled so that both a and b can work
08:50:48	*	shomodj joined #nim
08:51:21	federico3	arnetheduck: backward compatiblity works well for 99.9% of the cases. There are occasional bugs and you are cherry-picking one. Those cases can be handled as long as people still make an effort.
08:51:27	FromGitter	<arnetheduck> it's easier to do all that if it doesn't become so much of a problem to begin with, and that means battle-testing stuff *before* it goes into std
08:52:10	FromGitter	<arnetheduck> lol raymond chen of microsoft would probably have a different opinion :)
08:52:21	Araq	that sometimes works, and sometimes doesn't. KDE 4 only became stable long after v1 because before that not enough people tested it
08:52:30	Araq	er
08:52:34	Araq	I mean after 4.0
08:53:48	federico3	"grow worse over time as people start relying on them" -> that's a form of stability and it's a feature. When a bug is fixed in a way that changes an expected behaviour it's treated as a breaking change and warrants a new release train
08:54:47	*	tiorock joined #nim
08:54:47	*	tiorock quit (Changing host)
08:54:47	*	tiorock joined #nim
08:54:47	*	rockcavera quit (Killed (hitchcock.freenode.net (Nickname regained by services)))
08:54:47	*	tiorock is now known as rockcavera
08:57:44	FromGitter	<arnetheduck> well yeah, and packages offer exactly that: you want stability? use version 3.42 with the bug. you want the fixed version so that you don't have to pile more workaround on top of the bugs? go with the next version. best of both worlds - those that want no change are served and those that are actively maintaining their stuff get to do so. if backwards compatibility "happens" and the upgrade is easy, that's a
08:57:44	FromGitter	... fantastic bonus, but at least you have a "regular" way to deal with the breakage that can and will happen, every time
08:58:51	FromGitter	<arnetheduck> if you don't anticipate that stuff breaks, you're out of tune with reality and the tool/environment/langauge fails you when you actually need it
08:58:56	federico3	Araq: in general I would recommend announcing a guaranteed minimum time between a breaking change is introduced in a nightly and when it lands in a new release.
08:59:22	Araq	federico3, usually we provide a switch instead
08:59:38	Araq	but it depends on the kind of breaking change
09:00:19	federico3	Araq: yes, the switch is not always effective
09:01:12	federico3	arnetheduck: yes, that's how semantic versioning works
09:01:17	*	fjellfras joined #nim
09:01:49	*	fjellfras quit (Max SendQ exceeded)
09:02:07	FromGitter	<arnetheduck> switch doesn't scale for larger projects (like ours) because you end up with package a needing the switch and package b not.
09:02:21	*	fjellfras joined #nim
09:03:15	FromGitter	<arnetheduck> semantic versioning gives the author a way to communicate their belief about what breaks and what doesn't between versions. it's excellent! but as a consumer, I also need tooling to deal with when that belief turns out to be unfounded
09:03:52	FromGitter	<arnetheduck> *a structured way
09:04:20	federico3	also, a new breaking change can land into devel at any time with no warning and releases are not announced in advance. This forces developer to either test constantly against devel and rush to fix breakages or give up and let things break on a new release
09:04:54	*	absolutejam joined #nim
09:08:33	FromGitter	<arnetheduck> a new breaking change can be discovered in a release minor-point version and not announced in advance!
09:08:34	Araq	arnetheduck: well a switch is not an absolute solution, but a help for migration
09:09:04	Araq	and it did work for nimble and others fwiw
09:09:17	Araq	I'm talking about --nilseqs:on
09:09:19	FromGitter	<arnetheduck> sure, I'm just pointing out that things like that don't scale
09:09:30	*	absolutejam quit (Ping timeout: 258 seconds)
09:09:38	Araq	nothing "scales". For Google only their mono-repo scales.
09:10:20	FromGitter	<arnetheduck> if we're talking about bringing nim to more developers out there, we'll see more and more of these little tricks break apart.. that's why we're having this discussion now, so that perhaps we can anticipate how to handle it
09:10:50	Araq	I think it's an unhealthy word, it's wiser to think about a spectrum of pain
09:10:55	skrylar[m]	yup. i develop on a point release and ignore devel generally
09:11:16	Araq	skrylar[m], and that's what you should do.
09:11:35	skrylar[m]	sometimes i hook a CI up to do canary builds against devel
09:12:06	Araq	I also don't use "devel" versions of GCC, clang, Linux, ...
09:12:39	skrylar[m]	a lot of people target devel in nimble and nightly in rust
09:12:42	skrylar[m]	its odd
09:12:53	skrylar[m]	well for rust its often because a lot of things stay perpetually broken
09:13:06	Araq	if you want --newruntime, you should devel.
09:13:09	Araq	*use
09:16:22	federico3	Araq: do you see my point? People are given a binary choice: devel vs point release and nothing in the middle
09:16:43	Araq	no I'm sorry, too many people talking
09:17:39	Araq	what's the alternative? how does the "middle" look like?
09:18:54	FromGitter	<zah> I would personally appreciate more frequent point releases. Perhaps this would allow me convince @arnetheduck that we can finally spend a little bit of time investigating and fixing the bugs that we encounter
09:19:11	skrylar[m]	well if its put like that, i can just stop talking :shrug:
09:19:44	FromGitter	<arnetheduck> haha, yeah, and frequent point releases are easier when... dumroll... nim is smaller :)
09:21:18	dom96	actually, if I recall correctly the --nilseq flag didn't work for Nimble because we ended up shipping a broken Nimble that only worked with that flag or something like that
09:21:34	Araq	dom96, don't ruin it :P
09:21:51	federico3	Araq: doing alpha releases (perhaps more frequently) that are in feature freeze. After a fixed amount of time an alpha release turns into a point release (unless there are critical bugs that still need fixing)
09:21:56	dom96	also, in your PR it might help to call out the packages that broke and explain what your thinking
09:22:01	dom96	the PR has no description
09:22:12	dom96	barely any discernible explanation of what it does
09:22:27	dom96	so if you want to know what to do ask the community
09:23:22	federico3	Araq: ...so that people know in advance when a pre-release comes out and how much time do they have to test their packages against it
09:24:18	FromGitter	<arnetheduck> the .0 release of each series is the alpha release :)
09:24:34	Araq	I'm not a fan of "alpha" or "beta" releases, if we break stuff in X.0.0 there should be a X.0.2
09:27:01	Araq	possibly related: https://herbsutter.com/2019/07/13/draft-faq-why-does-the-c-standard-ship-every-three-years/
09:27:40	Araq	he argues for a fixed release schedule
09:28:49	Araq	> There are bugs in the standard, so should we delay C++20? -
09:28:49	Araq	Of course, and no.
09:30:16	*	shomodj quit (Quit: My MacBook has gone to sleep. ZZZzzz…)
09:36:12	FromGitter	<arnetheduck> pretty good article, yeah. shipt stuff that works, cut the stuff that doesn't (yet), have staging area for stuff to grow in before putting in std
09:39:45	federico3	Araq: call it odd/even then, but the point is to let library maintainers know in advance when things will break
09:40:06	FromGitter	<arnetheduck> fwiw, we recently opened a staging area of our own where we'll be trying out stuff before potentially proposing them for inclusion: https://github.com/status-im/nim-stew - also so that we can be a bit more independent of the point-release cycle and to some extent offer a migratory compatibility layer
09:43:15	Araq	yeah I've seen it, good stuff.
09:43:58	Araq	is it too late to change the name?
09:45:50	*	skaruts quit (Remote host closed the connection)
09:46:43	Araq	nim-boost or nim-turbo, maybe?
09:48:53	livcd	i propose "nimbo"
09:51:48	*	Vladar joined #nim
09:57:01	FromGitter	<arnetheduck> @zah minted it, you'd have to convince him :)
09:57:50	FromGitter	<zah> o.O what did I invent?
09:58:26	FromGitter	<arnetheduck> the `stew` name
10:00:02	*	fjellfras quit (Ping timeout: 248 seconds)
10:04:19	*	absolutejam joined #nim
10:04:42	Araq	livcd, I like "nimbo" too
10:05:13	FromGitter	<zah> ah, our hodge podge library. ⏎ ⏎ reminds me to ask, what are the reasons for why our proposal for a generic `init` scheme is not being accepted in the standard library?
10:07:45	Araq	I haven't rejected it yet, but in the meantime tables and seq default to the 'empty' value, mitigating the same pains your RFC addresses
10:08:45	Araq	I don't think Nim needs yet another way of writing initTable as annoying as that might be.
10:09:25	Araq	constructors didn't save C++ from introducing make_shared.
10:10:27	*	absolutejam quit (Ping timeout: 258 seconds)
10:14:46	dom96	Starting to think more and more that I'll need to write my own nimpretty implementation to get everyone to use the style I like :P
10:18:56	FromGitter	<zah> Araq, it's not only about tables though. We have a good deal of types that have init arguments and quite a lot of generic code that benefits from this
10:19:12	Araq	you'll enjoy getting Nim's comment handling right with its plethora of ways to write the same thing
10:19:35	dom96	Araq, Just make it consistent?
10:25:22	Araq	sure, write your own nimpretty, that's all I'm saying.
10:27:47	Araq	teach a system without eyes how to make code visually appealing, how hard can it be? :P
10:31:15	federico3	even better if we can configure the style
10:34:27	*	shomodj joined #nim
10:39:08	Araq	zah: ok, fair enough. I remember you told me about some issue though
10:39:40	FromGitter	<zah> there is an issue with perfect `varargs` forwarding. I'll file it
10:40:15	*	absolutejam joined #nim
10:40:34	*	solitudesf quit (Ping timeout: 246 seconds)
10:40:37	Araq	ah "prefect forwarding", C++ here we come, watch out :-)
10:41:04	FromGitter	<zah> It's easier in Nim, but not fully working :)
10:42:01	dom96	federico3, one true style might be better, it is what people love about Golang after all
10:42:47	federico3	no, Nim style insensitivity is a feature
10:43:10	Araq	we killed that.
10:43:20	federico3	what?!
10:43:36	Araq	it's --styleCheck now
10:44:23	narimiran	dom96: ...as long as that style is your style? :P
10:45:03	narimiran	it is also what people like about python's "black"
10:45:32	narimiran	yeah, some people don't like it, but it has generally been accepted, and it cuts down pointless discussions
10:45:48	*	absolutejam quit (Ping timeout: 268 seconds)
10:45:54	federico3	accept what?
10:45:58	Araq	federico3, well I was kidding, relax, it's only a linter, opt-in
10:46:01	narimiran	...so people now have more time for other pointless discussions :D
10:46:12	Araq	what's python's "black"?
10:46:18	*	Vladar quit (Ping timeout: 245 seconds)
10:46:22	federico3	Araq: like gofmt for python, but more relaxed
10:46:28	narimiran	https://github.com/python/black
10:49:21	FromGitter	<zah> Araq, I had already filed it actually: ⏎ proc foo(a, b: string, i = 10): string = ⏎ "foo called with " & a & "," & b & "," & $i ⏎ ⏎ proc bar(i = 10): string = ... [https://gitter.im/nim-lang/Nim?at=5d2efd313be99c4786673399]
10:49:47	FromGitter	<zah> Sorry, copy/pasting mistake. Here it is: ⏎ https://github.com/nim-lang/Nim/issues/9996
10:51:01	Araq	krux02 has some RFC pending for varargs ... *cough*
10:51:26	*	Vladar joined #nim
10:51:32	Araq	I wasn't aware we support this way of forwarding
10:51:33	*	absolutejam joined #nim
10:51:51	Araq	I would have used a macro with unpackVarargs
10:52:03	Araq	or whatever I called it
10:52:25	dom96	Araq, what? You killed style insensitivity?
10:52:36	Araq	I was kidding
10:52:41	Araq	but we now have a linter
10:52:54	Araq	and the *compiler*/nim.cfg enforces it
10:52:55	dom96	my god, you need to say you're joking after your joke
10:53:07	dom96	otherwise you're spreading misinformation
10:53:08	krux02	actually we don't support this way of forwarding.
10:53:45	FromGitter	<zah> varargs forwarding is supported through the use of `nkArgList` inside the compiler
10:53:46	dom96	black is in fact what FB uses (it was written by an ex-colleague)
10:54:33	dom96	narimiran: we should have a few discussions about the one true style and come up with some compromises
10:54:49	Araq	but we do have NEP-1
10:54:55	Araq	and nimpretty tries to follow it
10:55:01	dom96	I'm not arrogant enough to expect everyone to abide by every one of my style choices, but we should at least come up with something we can all agree to
10:55:06	Araq	we had this discussion!
10:55:10	dom96	NEP-1 was never properly scrutinized
10:55:27	Araq	it needs an update, PRs are welcome
10:55:48	federico3	that applies to the compiler and stdlib but I want to have a different style in libraries and C wrappers
10:56:05	Araq	federico3, sure and Nim lets you have that
10:56:38	federico3	hence my need for a formatter that allows configuring styles
10:58:50	Araq	that's not nimpretty's job
11:00:09	FromGitter	<alehander42> i also like the init idea
11:00:16	FromGitter	<alehander42> otherwise its hard to agree on a style
11:00:18	*	Vladar quit (Ping timeout: 248 seconds)
11:00:42	FromGitter	<alehander42> there will be always be cases that seems bad if one style is enforced
11:00:51	FromGitter	<alehander42> usecases*/snippets
11:00:54	FromGitter	<alehander42> that look bad*
11:04:26	FromGitter	<zah> `unpackVarargs` is not quite good, because you cannot insert additional arguments besides the ones you are unpacking
11:06:52	FromGitter	<zah> surely, you can write a one-off macro with a body similar to`unpackVarargs`, but why should things be that complicated
11:06:52	*	Vladar joined #nim
11:09:48	*	Vladar quit (Read error: Connection reset by peer)
11:10:58	Araq	er ... a take the 4 line macro over yet-another compiler feature including a node kind, nkArgList that I constantly need to remember for every transformation that I do
11:11:26	*	dddddd joined #nim
11:11:33	Araq	it's an unfair comparison because nkArgList already exists, but still
11:11:52	federico3	any help on https://github.com/FedericoCeratto/nimfmt is welcome
11:12:35	*	Vladar joined #nim
11:29:50	*	Ivo quit (Quit: Leaving)
11:30:05	FromGitter	<zah> If you remove the built-in support for `nkArgList`, it's possible to define `unpackVarargs` along these lines: ⏎ ⏎ ```code paste, see link``` ⏎ ⏎ Then it would be able to mix the varargs will other injected arguments [https://gitter.im/nim-lang/Nim?at=5d2f06bdd14e297762e94c78]
11:48:48	FromGitter	<zah> To support the `init` proposal properly, you'll need to use perfect forwarding in the system module though. Since macros are off-limit there, this means we either have to fix `nkArgList` inside the compiler or add `unpackVarargs` as a magic
11:52:43	*	zahary joined #nim
12:00:09	Araq	there is no reason system.nim cannot use macros
12:00:35	Araq	we don't do it but in the meantime system.nim imports plenty of other modules and it worked out
12:03:30	*	altarrel joined #nim
12:05:39	*	stefanos82 quit (Quit: Quitting for now...)
12:07:23	*	brakmic joined #nim
12:30:58	*	altarrel quit (Ping timeout: 248 seconds)
12:37:55	dom96	federico3, why semicolons in arg list, why :/
12:40:25	federico3	huh?
12:40:40	dom96	in nimfmt's output
12:41:08	federico3	that came out from Nim's parser/formatter. nimfmt is a very thin layer on it
12:44:10	FromGitter	<kayabaNerve> I just want a Nim linter to tell me of unused imports. I don't believe any Nim software does that yet.
12:44:17	*	sniffdtek joined #nim
12:44:47	FromGitter	<kayabaNerve> As a side note, going to ask this again, why is ProveInit a thing and can I disable it?
12:45:38	FromGitter	<kayabaNerve> Because I have a try init except fatal and would rather complain about it than add the line to fix it.
12:46:30	FromGitter	<kayabaNerve> Jokes aside, I do believe it's more counter productive than productive to users if it can't be disabled.
12:51:34	disruptek	would you rather maintain a large stdlib for a large number of users or develop a fast-moving compiler for a small number of users?
12:54:50	*	altarrel joined #nim
13:02:05	Araq	kayabaNerve: will do some livecoding session about it
13:02:17	Araq	I'm always looking for the easy stuff for these :P
13:03:04	narimiran	hint on unused imports? yes please
13:03:47	*	rockcavera quit (Remote host closed the connection)
13:04:20	*	UNIcodeX joined #nim
13:15:18	FromGitter	<kaushalmodi> > hint on unused imports? yes please ⏎ ⏎ +1
13:21:06	shashlick	what's the preference? need votes - https://github.com/nim-lang/Nim/issues/10630#issuecomment-511995621
13:25:21	Araq	as I said, use a 'select' template to cut the boilerplate
13:27:33	shashlick	not sure what that means which is why i put some examples up
13:27:53	shashlick	is it option 2? or somethingelse
13:31:10	Araq	something else
13:31:46	Araq	const CurDir = selectMac(':', '.') ## doc comment
13:32:21	Araq	you need a couple of selectMac/Posix/Doslike variants. so maybe it's not worth it
13:32:31	Araq	and your 'when' is better
13:32:41	Araq	(that would be option 1 then)
13:34:45	*	ofelas quit (Quit: shutdown -h now)
13:35:45	shashlick	okay option 1 it is, will create a PR in some
13:39:00	Araq	disruptek: you raise a good point. We're talking without any numbers.
13:39:14	*	rockcavera joined #nim
13:40:18	*	rockcavera quit (Remote host closed the connection)
13:40:38	*	rockcavera joined #nim
13:43:08	disruptek	i really don't know what the preference is for the community, but it doesn't have to be inclusive. maybe you should light a torch for a compiler designer and focus on the stdlib if that's what you want to invest in.
13:44:49	Araq	well I'm about to have holidays
13:44:58	Araq	I'll go inside me and think about what to do
13:47:28	*	UNIcodeX quit (Quit: Leaving)
13:49:37	disruptek	shashlick: well, i couldn't automate cpp codegen. if you have a few minutes to look at a wrapping attempt, i'd appreciate it.
13:50:06	shashlick	Am in a two day secure coding training
13:50:18	shashlick	But please text - I'll catch up when I can
13:50:43	shashlick	Araq - does https://arxiv.org/abs/1907.01933 influence your Nim cold fusion idea
13:50:50	disruptek	it's aws-cpp-sdk or aws-sdk-cpp, depending on where you are in the repository.
13:51:01	shashlick	App specific compiler stack
13:51:24	shashlick	Can you share a link @disruptek
13:52:19	disruptek	shashlick: sure, but i won't blame you if you take a pass: https://aws.amazon.com/sdk-for-cpp/
13:52:33	Araq	shashlick: I'm aware of it but it doesn't influence me wrt Nim cold fusion
13:53:57	shashlick	Cool
13:54:03	Araq	lol
13:54:25	disruptek	i am gonna try to wrap this for two reasons: it'll be a big win for nim, and the code is pretty damned tame. once we have a solution, we can take our time at a new codegen effort. i'm not sure i can use ryukoposting's work.
13:54:31	shashlick	@disruptek @varriount and one other person were looking at an aws wrapper
13:54:36	shashlick	Was it you or someone else
13:55:11	disruptek	yes? but i hadn't done anything but run the tests. now i come to find that half the product is unimplemented.
13:55:56	shashlick	I see - well, i'm happy to help
13:56:31	shashlick	i'd use nimterop for everything except the actual wrapping and call c2nim underneath if the headers are c++
13:56:49	shashlick	nimterop needs some work to integrate c2nim
13:57:05	shashlick	now that c2nim is more backwards compatible, i might consider making it a dep
13:57:15	shashlick	but ideal way would be for wrapper to depend on nimterop and c2nim
13:57:18	disruptek	actually, that might be the best solution. as i said, the code is pretty tame.
13:57:59	disruptek	i struggle to follow the api generator, which is in nim, if that gives you a sense of my ability level.
13:58:18	shashlick	c2nim isn't fond of advanced templates, tho @ratiotile has made some PRs
13:58:19	disruptek	or of ryukoposting's ability to obfuscate...
13:58:48	shashlick	okay i'm game to help out with this
13:59:10	shashlick	but obviously very distracted right now - what specific API do you want to start with
13:59:39	disruptek	i'm trying to invoke() a lambda function from the ...-lambda/ tree.
14:00:40	FromGitter	<alehander42> hm what about nim-extended
14:00:42	disruptek	but basically, the problem i have is that the most common header, aws/core/aws.h cannot be toasted.
14:00:42	shashlick	do you intend using the nimterop API (cImport, etc) dynamically or generate wrappers using the CLI
14:00:45	FromGitter	<alehander42> cold-fusion was taken right
14:01:14	disruptek	i've tried both ways, and it seemed like i got farther in cimport, but it wouldn't run in cpp for me, so i went back to toast.
14:01:45	disruptek	i had to hack some coldfusion once. Once.
14:02:44	shashlick	what error did you have with aws.h
14:02:59	shashlick	use toast to preprocess then run through c2nim --cpp
14:03:42	disruptek	Error: unhandled exception: getters.nim(108, 12) `name.len != 0` Blank identifier error [AssertionError]
14:03:47	Araq	please don't run an external preprocessor, c2nim ships with one
14:04:26	shashlick	that requires code changes to the h files
14:04:38	disruptek	how rude.
14:05:45	shashlick	@disruptek - that error looks like an issue when trying to convert to nim, nimterop doesn't know c++
14:05:59	shashlick	you just want to use -p to preprocess
14:06:11	disruptek	i think it needs to know about namespaces, no?
14:06:33	shashlick	nimterop doesn't know anything in C++ - classes, templates, namespaces, etc.
14:06:38	shashlick	you have to use c2nim
14:07:19	shashlick	doesn't look like aws.h uses much in terms of preprocessor stuff
14:07:25	shashlick	so you could even skip toast
14:07:26	disruptek	what does the cpp mode do?
14:07:40	Araq	it enables c2nim's C++ support
14:08:10	shashlick	i think he's asking with nimterop - all it does now is parses the code as cpp using tree-sitter, i guess it is confusing
14:08:31	shashlick	nimterop cannot convert cpp => nim
14:08:37	disruptek	yes, especially because it seemed that i got better fidelity when switching to cpp mode.
14:08:59	FromGitter	<dwlnetnl> Hello, I am wondering if it's possible to build a static C library written in Nim
14:09:01	shashlick	that's why i have it available
14:09:19	shashlick	dwlnetnl - yes, look at exportc
14:09:33	disruptek	i just tested it and cpp mode crashes and mode=c works fine. so, that's worse fidelity.
14:09:55	disruptek	but, clearly that mode change is impactful.
14:10:04	FromGitter	<dwlnetnl> shashlick: oh yes that's cool
14:10:21	FromGitter	<dwlnetnl> can you also use exportc to export types etc?
14:10:25	FromGitter	<dwlnetnl> or just procs
14:10:38	disruptek	anything.
14:10:43	disruptek	more or less the same as c.
14:10:43	shashlick	again, it controls which tree-sitter engine i use, it isn't crashing but complaining about the naming used
14:11:17	shashlick	anyway, is a distraction for what you want to do - you need to use c2nim for c++ headers
14:11:20	disruptek	well, this probably solved my problem, thank you.
14:11:26	FromGitter	<dwlnetnl> And how would it work memory wise? I see that Nim uses a GC by default
14:11:49	FromGitter	<dwlnetnl> can you like 'disable' the GC?
14:11:53	disruptek	there are alternative arrangements available. more than you might imagine.
14:12:04	FromGitter	<dwlnetnl> so it's easier in C and other languages?
14:12:08	disruptek	yes, you can disable it, swap it, omit it.
14:12:39	FromGitter	<dwlnetnl> but you still seems to need to call NimMain, do you?
14:13:18	disruptek	i imagine so, if you wanted to build a static lib that uses a gc.
14:13:37	disruptek	someone else here could surely answer for sure, surely.
14:14:31	FromGitter	<dwlnetnl> I'd probably want to build a static c lib in a way that is as easiest to interoperate/embed in other languages
14:14:51	FromGitter	<dwlnetnl> maybe reference counted?
14:15:39	disruptek	i think you should install the compiler and dig through `--fullhelp` to identify qualities you deem important to your application.
14:16:04	FromGitter	<dwlnetnl> sure, I'll have a look
14:16:09	FromGitter	<dwlnetnl> thanks for the help so far
14:16:53	*	stefanos82 joined #nim
14:17:21	disruptek	sure. see the --gc option; refc, mark&sweep, boehm, go, (none), or regions. lots of options and as i said, more than you might imagine.
14:17:51	shashlick	https://nim-lang.github.io/Nim/manual.html#foreign-function-interface-exportc-pragma
14:18:26	shashlick	https://nim-lang.github.io/Nim/gc.html
14:19:25	*	altarrel quit (Quit: Quit)
14:19:42	*	altarrel joined #nim
14:20:42	FromGitter	<mratsim> there was a recent article on building static lib Nim that was quite good
14:20:45	FromGitter	<mratsim> with musl
14:29:56	FromGitter	<mratsim> https://www.google.com/search?q=nim%20musl%20static%20binary
14:30:07	FromGitter	<mratsim> sorry: https://scripter.co/nim-deploying-static-binaries/
14:30:22	disruptek	thanks. ;-)
14:30:38	FromGitter	<mratsim> @dwlnetnl ^
14:37:29	FromGitter	<dwlnetnl> thanks@!
14:41:52	disruptek	ah, that's our own kaushalmodi; i never put that together before. :-)
14:56:52	FromGitter	<yglukhov> Hey guys. Was there any discussion about renaming Exception -> RootError, CatchableError -> Exception, and changing `except:` semantics to `except Exception:` instead of `except RootError`?
14:57:47	disruptek	that's probably something that should have gone into 0.20.
15:01:17	*	UNIcodeX joined #nim
15:01:47	FromGitter	<yglukhov> If we treat `except:` being equal to `except RootError:` as a bug, there's still a way to do it :).
15:03:03	disruptek	there aren't too many other bugs i can imagine breaking quite so much as the fix for this one.
15:04:10	FromGitter	<yglukhov> why? Defects were introduced quite recently for exactly the reason to not being easily caught, no?
15:04:46	disruptek	because exceptions are control flow whether we like it or not and you're changing the semantics under the covers.
15:05:27	FromGitter	<mratsim> we can have a -d:nimOldExceptions
15:06:15	disruptek	you want to introduce a compile-time switch to restore old behavior as a bugfix release?
15:06:24	disruptek	good god, are you mad, man?
15:06:44	Araq	yglukhov: can't we fix 'except:' without renaming the exception types yet again?
15:06:45	disruptek	what will you write your code in? new style or old?
15:06:54	Araq	disruptek: please calm down
15:06:57	FromGitter	<arnetheduck> exceptions are random and hidden control flow anyway :)
15:07:05	disruptek	i'm just kidding aroun'. ;-)
15:07:22	FromGitter	<yglukhov> Araq: we can, but if you're into it, you
15:07:58	FromGitter	<yglukhov> 're breaking old code already. at which point we should think if we can do it right once and for all :)
15:08:13	*	actuallybatman joined #nim
15:08:34	FromGitter	<yglukhov> And then CatchableError can remain an alias to Exception for backward compat
15:08:39	Araq	I think I heard that one before, "this will be the last breaking change..."
15:08:47	FromGitter	<mratsim> famous last words :p
15:08:54	disruptek	hey, i don't disagree that it needs fixin'. i just don't think it's small enough to go into a patch release.
15:09:03	FromGitter	<arnetheduck> re `init` - I still miss the ability to use them to balance destructors and enforce their use for a particular type. without that, they're just syntactic sugar
15:10:13	*	absolutejam quit (Ping timeout: 258 seconds)
15:11:00	disruptek	this is why i wanted `exception MyError of ValueError as e:` syntax.
15:12:50	disruptek	call it 0.21 and then people can opt-in in a sane way. no reason not to have a 0.21 branch, right?
15:13:09	Araq	we have this branch already, it's called 'devel'
15:13:14	FromGitter	<yglukhov> "this will be the last breaking change" every change is breaking. It just depends where you draw the line :).
15:13:46	Araq	well write a short RFC about it then
15:13:54	FromGitter	<yglukhov> alright
15:13:55	Araq	or find some existing
15:14:10	disruptek	if you break 0.20 you can unbreak it. if you break 0.21, you just re-break it correctly.
15:14:14	FromGitter	<yglukhov> found none, that's why I asked
15:14:41	*	floppydh quit (Quit: WeeChat 2.5)
15:14:43	disruptek	there's an existing issue for this.
15:15:27	FromGitter	<yglukhov> oh? which one?
15:16:02	*	sagax quit (Quit: Konversation terminated!)
15:16:16	*	sagax joined #nim
15:16:34	disruptek	i think the one i'm thinking of is: https://github.com/nim-lang/Nim/issues/10288
15:17:32	*	xet7 quit (Ping timeout: 245 seconds)
15:17:36	Araq	we can declare it a bug and fix it in 0.20.4
15:18:20	Araq	system.Defect is not catchable, 'except:' shouldn't catch it. Er .. and break the unit testing stuff.
15:19:25	FromGitter	<yglukhov> disruptek: yeah, I've seen that, but it looks different.
15:19:52	disruptek	how is it different? the semantics need to reflect the names.
15:20:02	disruptek	that's the bug in a nutshell.
15:20:22	disruptek	otherwise, we don't know how to write a compiler that can intuit a programmer's intention.
15:20:46	FromGitter	<yglukhov> Araq: good point about unit testing.
15:22:01	FromGitter	<yglukhov> disruptek: I'm saying 10288 is orthogonal to what I'm suggesting. It talks about Exception subclasses and their inheritance.
15:23:47	FromGitter	<GULPF> there's also https://github.com/nim-lang/RFCs/issues/77 which is related as well
15:23:55	disruptek	if you're not talking about changing the semantics, then i don't know what you're talking about, to be honest.
15:24:43	*	nsf quit (Quit: WeeChat 2.4)
15:26:28	FromGitter	<yglukhov> @GULPF thanks
15:28:00	FromGitter	<arnetheduck> I've mostly learned to ignore bad naming, but oh boy is CatchableError ugly
15:28:10	*	lritter joined #nim
15:28:14	FromGitter	<zah> it would have been nice if CatchableError was named just Error IMO
15:28:18	FromGitter	<kaushalmodi> Just saw a mention of my blog post above.. it's been a while since I wrote a post..
15:29:02	Araq	huh?
15:29:15	Araq	what's bad about that name?
15:29:22	FromGitter	<zah> The base Exception type could have a name that nobody in their right mind would use - for example StackUnwindingEvent
15:29:41	FromGitter	<zah> In our team, people inherit from Exception too often because they forget about the distinction
15:29:43	federico3	isn't "catchable" implicit?
15:29:51	Araq	it's a catchable error, it doesn't get clearer than that
15:30:06	FromGitter	<zah> Error - short and simple :)
15:32:50	*	altarrel quit (Ping timeout: 248 seconds)
15:33:32	disruptek	maybe except: could set the exception type to Error instead of Exception. now you have a differentiator that's at least a little less painful. i really hate doing this as a bugfix though.
15:34:57	Araq	disruptek: yeah I take it back, this is not a bugfix
15:35:15	*	sentreen_ joined #nim
15:36:22	Araq	you shouldn't inherit from CatchableError btw
15:36:35	Araq	usually it's IOError, OSError or ValueError
15:37:28	Araq	the name would be bad if it's what you should use, but you shouldn't use it
15:37:31	disruptek	that makes sense.
15:38:02	*	sealmove joined #nim
15:38:04	*	sentreen quit (Ping timeout: 246 seconds)
15:38:47	FromGitter	<zah> hrm, we've adopted the practice of introducing a root cachable error exception for each subsystem (e.g. SerializationError) and then having more-specific sub-types
15:39:25	FromGitter	<zah> picking one of the std lib errors as a base is not appropriate with this approach
15:39:28	*	sentreen joined #nim
15:39:44	disruptek	i guess you need a variant exception type that can hold ioerror, oserror, or valueerror -- if you want inheritance for your exception types.
15:41:59	Araq	zah: since only OSError has the errorCode*: int32 field it's what you should have inherited from afaict
15:42:32	*	sentreen_ quit (Ping timeout: 245 seconds)
15:43:18	*	Trustable joined #nim
15:43:34	Araq	bbl
15:44:10	FromGitter	<zah> SerializationError may mean "missing comma in a json file", "RLP object with an invalid blob inside" and so on. Having a base type makes it easy for you to just treat all such errors the same (the details are irrelevant in many situations)
15:44:25	FromGitter	<zah> The same pattern is present in other sub-systems
15:44:34	disruptek	preaching to the choir, my dude.
15:46:45	FromGitter	<rishavs> Is there any article, tut on using the peg module?
15:54:56	*	sealmove quit (Quit: WeeChat 2.5)
15:56:43	FromGitter	<alehander42> just askk Zevv
15:56:46	FromGitter	<alehander42> oh wait
15:56:49	FromGitter	<alehander42> sorry nvm
15:56:53	FromGitter	<alehander42> i thought npeg
15:56:58	FromGitter	<rishavs> :D
15:59:50	FromGitter	<rishavs> i am assuming that npeg is the port of lua lpeg?
16:03:04	Zevv	basically, although it compiles to nim code instead of executing the peg in a vm
16:03:52	Zevv	and it has some limitations conpared to lpeg involving captures, because of the static typing
16:04:28	FromGitter	<rishavs> cool. will check out
16:06:56	*	brakmic_ joined #nim
16:07:45	*	brakmic_ quit (Client Quit)
16:08:04	*	brakmic_ joined #nim
16:08:10	*	brakmic quit (Read error: Connection reset by peer)
16:19:02	*	sealmove joined #nim
16:22:17	Araq	zah: well JsonError inherits from ValueError, it's nothing new
16:23:58	FromGitter	<zah> Perhaps I picked a bad example, but defining a complete taxonomy of errors in the stdlib seems like a tall order for me. I don't see a problem with people inheriting from `Error` directly instead
16:34:57	Araq	inherit from ValueError, OSError or CatchableError but I don't think CatchableError is badly named
16:35:23	Araq	it's not the "don't have to think" solution, it doesn't need a nice name
16:49:33	*	altarrel joined #nim
16:52:29	FromGitter	<rishavs> @araq some questions on compiler design; I am trying to create a toy language which will compile down to c. I have a good idea of parsing but I am not sure what to do after AST generation. ⏎ ⏎ 1) Is there a specific format I should adhere to? ⏎ 2) Is there a tool/lib that you would recommend to convert ast to c code? ⏎ 3) I want to have GC in the future. is there anything specific that i should keep in mind
16:52:30	FromGitter	... right now when I am still deciding on the toolchain? [https://gitter.im/nim-lang/Nim?at=5d2f524d01621760bcd03f95]
16:56:35	*	brakmic_ quit ()
16:56:58	*	ibutra joined #nim
16:57:01	Araq	one "trick" that you should do (and Nim doesn't) is that you should produce the C code as a tree, don't do string concats everywhere
16:57:21	FromGitter	<zah> While we are discussion exceptions there are few other things I've learned with experience: ⏎ ⏎ 1) The built-in `msg` string field of exceptions feels a bit sub-optional at times. I'd like to create exceptions that carry light-weight data and that will be rendered as strings only sometimes. We could have that if `msg` was a method instead of a field ⏎ ⏎ 2) Exception chains are nice thing to have, because they
16:57:21	FromGitter	... don't ask me to choose between a complete stack trace and most appropriately typed exception (I'm referring to the situations where you want to remap some system exceptions to your own sub-system exceptions, so you catch them and then reraise as another type) [https://gitter.im/nim-lang/Nim?at=5d2f5371f4fe943971260003]
16:57:55	*	shomodj quit (Quit: My MacBook has gone to sleep. ZZZzzz…)
16:57:57	*	rokups quit (Quit: Connection closed for inactivity)
16:59:01	FromGitter	<zah> opps, *feels a bit sub-optimal*
16:59:15	Araq	about the GC: avoid everything with the name 'conservative' in it :-)
17:02:31	FromGitter	<rishavs> thanks
17:03:02	*	stefanos82 quit (Quit: Quitting for now...)
17:04:48	Araq	another tip: C typedefs are a pita, easiest seems to be to emit them and then re-order them in a postprocessing step
17:05:16	Araq	(Nim doesn't do that either :-) )
17:06:09	Araq	zah: nothing stops you from not using the 'msg' field and to use a method instead
17:06:45	Araq	(But I agree)
17:06:50	FromGitter	<zah> that's what I do, but then my exceptions don't play nice with the built-in handling
17:09:30	*	nsf joined #nim
17:12:42	rayman22201	@dom96: you around? More async questions. Does the async wait forever on a select() or poll ()? Or does it periodically timeout to check the work queue?
17:14:55	*	Trustable quit (Remote host closed the connection)
17:15:37	shashlick	@kaushalmodi - u around?
17:16:30	*	absolutejam joined #nim
17:23:11	*	zyklon joined #nim
17:23:48	*	shomodj joined #nim
17:26:57	*	altarrel quit (Quit: Leaving)
17:27:39	*	uvegbot quit (Ping timeout: 264 seconds)
17:34:02	*	ibutra quit (Quit: My MacBook has gone to sleep. ZZZzzz…)
17:34:51	*	ibutra joined #nim
17:36:31	*	solitudesf joined #nim
17:40:49	*	absolutejam quit (Ping timeout: 268 seconds)
17:40:59	FromGitter	<kaushalmodi> shashlick: yes
17:42:12	*	ibutra quit (Quit: My MacBook has gone to sleep. ZZZzzz…)
17:42:39	*	ibutra joined #nim
17:44:01	*	ibutra quit (Client Quit)
17:45:59	dom96	rayman22201, asyncdispatch.poll takes a timeout
17:47:12	*	oculux quit (Quit: blah)
17:48:30	*	oculux joined #nim
17:55:52	shashlick	@kaushalmodi - looks like docs are deployed by every travis job rather than just one - https://github.com/nim-lang/Nim/blob/devel/.travis.yml#L78
17:56:01	*	altarrel joined #nim
17:56:34	*	altarrel left #nim (#nim)
17:56:38	leorize	yea, someone should limit that to osx
17:56:42	shashlick	might be better to have a condition so it happens only for one - https://github.com/genotrance/feud/blob/master/.travis.yml#L52
17:56:44	*	altarrel joined #nim
17:57:14	shashlick	i'm also looking into generating a docset using dashing and niv/nim-docset
17:58:18	shashlick	as part of travis so that it is also kept up to date
18:02:39	sealmove	how do you by-pass object field/function conflict?
18:02:43	FromGitter	<kaushalmodi> shashlick: hmm, somewhere I had a DO_DEPLOY env var setting.. may be that is set just once?
18:03:22	shashlick	that's in nightlies, not in devel
18:04:06	FromGitter	<kaushalmodi> right
18:04:15	FromGitter	<kaushalmodi> so devel would need something like that
18:04:16	FromGitter	<kaushalmodi> see https://github.com/nim-lang/nightlies/blob/master/.travis.yml#L106
18:04:23	*	artemis_coven joined #nim
18:04:44	shashlick	can just do it how i did it in feud - the second link I shared
18:05:06	leorize	oooh 0.20.2
18:05:35	FromGitter	<kaushalmodi> shashlick: but that `on:` condition is on Nim devel too
18:05:36	shashlick	OS = osx && NIM_COMPILE_TO_CPP = false
18:05:59	FromGitter	<kaushalmodi> so just tighten that on: condition
18:06:04	narimiran	leorize: yeah :)
18:06:21	shashlick	yep
18:06:23	narimiran	we're getting too casual with these releases :P "oh, btw, 0.20.2 is here"
18:07:35	*	shomodj quit (Quit: My MacBook has gone to sleep. ZZZzzz…)
18:11:34	shashlick	wish https://github.com/nim-lang/Nim/issues/11553 was fixed in 0.20.2
18:12:01	shashlick	but 200 issues in 40 days is amazing
18:12:13	narimiran	shashlick: we need to leave something for 0.20.4 ;)
18:12:37	shashlick	😄
18:12:47	shashlick	awesome work though
18:13:06	Cadey	you can pass a proc as a proc argument right?
18:13:29	shashlick	yes
18:13:33	narimiran	i'm personally impressed that we managed to focus mostly on bugfixes, and not just work on new (breaking) features. and i hope this will continue in the upcoming months
18:14:46	Cadey	hyped for nim 1.0
18:15:15	*	absolutejam joined #nim
18:16:53	rayman22201	@Dom96, thanks. That's what I thought.
18:18:00	sealmove	newSeq is called automatically for objects that contain seq fields now?
18:19:34	*	elrood joined #nim
18:24:55	Araq	sealmove: kinda
18:24:58	*	skaruts joined #nim
18:25:08	skaruts	what's the best data structure to use if I need to delete elements while keeping the order they're at?
18:25:30	skaruts	lists?
18:25:43	Araq	a seq
18:26:34	narimiran	OrderedSet?
18:30:13	skaruts	doesn't the seq move the last element to the empty place?
18:30:24	narimiran	skaruts: `del` vs `delete`
18:32:15	skaruts	hmm
18:32:42	skaruts	oh, I never knew that
18:32:50	skaruts	delete keeps the order
18:32:57	skaruts	thanks
18:40:38	artemis_coven	does cross compilation work from arm host architecture/
18:41:04	artemis_coven	I'm trying the cross-compilation listed in the docs and telling it to generate an amd64 binary, and its just generated an arm binary instead
18:41:33	Cadey	do you have an amd64 gcc toolchain installed?
18:41:37	artemis_coven	yeah
18:42:51	artemis_coven	maybe I need to manually configure the paths for the amd64 toolchain or something
18:45:59	artemis_coven	yeah that did it
18:46:05	dom96	Hope you guys tested both Nim and Nimble in the release this time :)
18:46:19	narimiran	dom96: :P
18:50:52	dom96	Nice to see a tweet as well
18:51:07	dom96	I think a separate forum thread is warranted for these releases, no need to bump the thread I created
18:51:16	dom96	but *shrug*
18:51:32	Araq	it's ok this way, it's only a bugfix release
18:53:49	*	altarrel quit (Quit: ZNC 1.6.5+deb1+deb9u2 - http://znc.in)
19:01:03	rayman22201	Sooo.... @Dom96 and @Araq, @Zevv just convinced me that having a timeout int Poll is really broken
19:04:09	dom96	Really? Strange that both POSIX and the WinAPI offer this timeout argument then, no? :)
19:04:59	rayman22201	those aren't async apis. any good sync api has a timeout, what does that have to do with async?
19:05:44	rayman22201	comparing to the chronos implementation. Chronos does not accept a timeout
19:06:17	rayman22201	It finds the lowest timer value, or if there are no timers, waits forever (until signaled).
19:07:53	rayman22201	which is correct. That is how it has to be, otherwise, you either spin the async event loop too much burning your cpu cycles, or you wait too long, having bad performance.
19:08:32	rayman22201	The current stdlib impl, defaults to waking up twice a second no matter what. This is not a great implemenation.
19:09:32	*	altarrel joined #nim
19:10:34	*	absolutejam quit (Ping timeout: 258 seconds)
19:10:54	dom96	The current stdlib implementation doesn't assume that you're happy to give complete control to the async event loop.
19:10:56	rayman22201	Looking at it another way, it would be impossible to implement the uniqueID asyncEvent trick we talked about in Chronos. It would deadlock. The same thread cannot both read and write to the same pipe, because it will block forever on the read, and never get a chance to do the write.
19:11:50	dom96	If this makes a real practical difference in benchmarks then I'll listen to you, but until then it's a small implementation detail
19:11:51	rayman22201	the trick will work in the stdlib, but only because of these "forced wakeups", which forces progress to be made.
19:12:32	rayman22201	Well, at the very least it is a huge design difference between Chronos and the stdlib, leading to even more fragmentation (which you are against.)
19:12:52	*	altarrel quit (Client Quit)
19:13:02	*	altarrel joined #nim
19:14:01	dom96	I don't see how these "forced wakeups" make a difference
19:14:08	rayman22201	At worst, it has bad efficiency issues. Especially on mobile. (@Zevv firmly believes this. Also Status made this decision very conciously, and they are smart people who want nim to target mobile... They care about this stuff. why would they do it otherwise?)
19:14:20	dom96	If you call `runForever` (which most apps do) you will be running `while true: poll(500)`
19:14:44	rayman22201	Think about a phone. Phones go into a low power mode. If they have to wake up every 500ms, it will kill your battery.
19:14:57	dom96	so increase the timeout?
19:15:07	dom96	Nothing stops you from passing -1 to the `poll`
19:15:14	dom96	which means "infinite"
19:15:44	rayman22201	True. But it is not the default. and defaults are important
19:15:48	*	absolutejam joined #nim
19:15:54	*	altarrel quit (Client Quit)
19:16:33	dom96	Changing it in `runForever` and `waitFor` should be safe
19:16:57	rayman22201	Also, back to the asyncEvent thing. If we implemented your suggestion and somebody sets the timeout to -1, asyncEvents will suddenly not work anymore.
19:17:09	dom96	Why not?
19:17:21	dom96	The master async event will get triggered and things will work
19:17:38	rayman22201	because of the deadlock I described. No. It will block forever on the read and never do the write to trigger the wakeup.
19:18:01	Araq	don't ask me, I believe in polling moreso than everybody else
19:18:18	rayman22201	If the same thread is holding both ends of the pipe, it cannot both wait forever on a read, and do a write.
19:18:23	Araq	it's the only thing that really works and composes well with all the other stuff
19:18:40	dom96	You keep saying `pipe` and I don't understand why, do you mean a conceptual pipe?
19:18:46	dom96	My solution doesn't involve any actual pipes
19:19:21	rayman22201	yes. I mean conceptually. a select() call followed by a trigger.
19:19:25	dom96	But I think I get what you mean
19:19:33	*	altarrel joined #nim
19:19:43	Araq	so ... I'm about to stream
19:19:50	rayman22201	I'm from the unix world, so pipes are familiar to me, but any trigger mechanism, IOCP, whatever.
19:20:01	dom96	Araq, how does the flowvar signal proc get scheduled?
19:20:23	Araq	it's based on a "condition variable", the OS does the scheduling
19:20:47	dom96	so if my thread is blocked on a select() call it will still get scheduled by the OS?
19:20:49	rayman22201	doesn't matter what flowvar does. If the main thread is doing `select(asyncEventFd, -1)` it will wait forever.
19:21:01	dom96	perhaps it will use some signal to break way from the select call?
19:21:14	rayman22201	that signal has to be in another thread.
19:21:23	dom96	I feel like you're being a little intentionally distrustful, I am just trying to help I hope you know
19:21:54	rayman22201	no! not at all. I'm sorry I'm coming off confrontational! I'm just excited and caffeinated. :-)
19:22:04	rayman22201	I think it's an interesting problem.
19:22:17	Araq	dom96, if select blocks the full thread is blocked
19:22:30	Araq	the OS can only schedule other threads to run
19:22:56	dom96	Araq, but how does this work? Presumably a thread is always doing /something/
19:23:10	dom96	The OS must tell it "pause for a second and go into this signal proc"
19:23:38	Araq	well the 'signal' wakes up the other thread that waited for this signal
19:23:51	Araq	I'm not sure I understand your question
19:24:16	dom96	Argh.
19:24:28	Araq	signal is paired with 'wait' aka 'blockUntil'
19:24:54	dom96	but we don't `wait` in the examples that we ran on the stream
19:25:14	dom96	we just have the main thread calling select() all the time
19:25:16	Araq	well but '^' does
19:25:21	dom96	yes, but we don't use that
19:25:31	rayman22201	we do. the read wraps '^'
19:25:34	Araq	we did use the roof in the callback, didn't we?
19:25:38	rayman22201	yup
19:25:40	dom96	We do, but only after the FlowVar completes
19:25:48	dom96	The signal proc gets called before we call ^
19:26:09	rayman22201	Right, this works because of the spurious `500ms` wakeups
19:26:13	rayman22201	not because of flowvar
19:26:19	Araq	hmmm, maybe
19:26:21	dom96	Are you sure?
19:26:28	dom96	Can you try it with -1 real quick?
19:26:40	rayman22201	good idea. I will test it
19:26:40	dom96	Even if that is the case, after the 500ms wake we call select() immediately after
19:26:50	dom96	there is no "threadpool.canYouPleaseCheckForCompletions()"
19:27:11	*	altarrel left #nim ("Leaving")
19:27:18	dom96	Araq, Do you understand my dillemma?
19:27:27	*	altarrel joined #nim
19:27:30	dom96	How can this `signal` proc be called in the main thread?
19:27:49	dom96	How can the OS schedule it _out-of-order_ (that's possibly the wrong term)
19:28:11	*	altarrel quit (Quit: ZNC 1.6.5+deb1+deb9u2 - http://znc.in)
19:31:22	Araq	kind of, maybe
19:31:48	Araq	I'm about to stream, today's topic: detect unused imports
19:32:20	dom96	cool
19:33:37	Zevv	i still do not understand what all the confusion is about :/
19:33:53	Zevv	i'm just waiting for the dust to settle
19:34:02	Araq	Zevv, that means you should answer the questions
19:34:16	dom96	Yes, please explain your point of view
19:34:56	Zevv	im on the road on mobile
19:35:04	Zevv	but i'll get back, sure
19:35:23	*	altarrel joined #nim
19:35:55	disruptek	i think the reason you're using a fd is so select can trigger on it. that's something the os does for you.
19:36:07	disruptek	not sure what else you are missing, tbh.
19:36:20	Araq	https://www.twitch.tv/araq4k
19:43:01	*	nsf quit (Quit: WeeChat 2.4)
19:48:21	shashlick	If I copyMem a string I'm not really copying the contents right? Just a pointer to the contents?
19:48:30	shashlick	How about cstring?
19:49:21	dom96	shashlick, depends how you pass it into copyMem
19:53:47	Calinou	is it just me or execProcess()' signature has changed twice recently?
19:53:57	Calinou	it seems to expect a workingDir argument again, while I specifically removed it last month
19:54:07	Calinou	https://github.com/Calinou/godot-size-benchmarks/commit/6dc999f14625936110cd980448d04515d512ff1c, that is
19:54:48	Calinou	yeah, it builds if I revert it. I'm using Nim 0.20.0
19:54:56	*	yeetcannon quit (Remote host closed the connection)
19:55:38	*	yeetcannon joined #nim
19:57:05	leorize[m]	there was a change to make `execProcess` have a similar signature with the rest of osproc iirc
19:58:56	Calinou	I see, it's a welcome change anyway, I'm just surprised it changed twice between two releases
20:01:38	leorize[m]	and it should still accept workingDir
20:01:50	leorize[m]	just that it's now in a different position
20:02:02	rayman22201	@Zevv, @Dom96, I'm totally confused now. Poll(-1) still works... but I don't understand how.
20:02:10	dom96	hehe
20:02:25	rayman22201	It is way more efficient to use Poll(-1) though :-P
20:02:31	rayman22201	no spurious wakeups
20:02:48	rayman22201	I don't understand how though
20:02:48	dom96	Probably. But I'm skeptical about "way more"
20:03:15	*	narimiran quit (Remote host closed the connection)
20:03:51	rayman22201	I guess it depends on your use-case. It does make me think that waitFor and runForever should accept optional timeout values though.
20:04:07	rayman22201	I can definitely see how it would affect mobile
20:04:34	disruptek	sounds like it's a bug that it works.
20:04:36	rayman22201	that's a bikeshed though... I really want to understand how this works. It's blowing my mind lol
20:04:46	leorize[m]	looks like there's still code for idetools in the compiler
20:04:50	leorize[m]	not sure if they're still used or just dead code
20:08:43	dom96	rayman22201, my guess: the OS interrupts the select(), then runs the `signal` function, or vice versa
20:08:55	*	jjido joined #nim
20:08:55	dom96	Add some echos around the place and you should find out
20:09:01	dom96	It is very interesting
20:09:24	dom96	But yes, whether we run poll with -1 or not is totally orthogonal. I'm glad we established this :)
20:11:36	Zevv	rayman22201: so, which part is not clear then?
20:12:47	rayman22201	what Dom said. "The OS interrupts the select(), and runs the signal function". In this case the signal function is a semaphore from the threadpool (I think?)
20:12:58	Zevv	right
20:13:23	dom96	Do you know how the scheduling of this `signal` function is performed Zevv ?
20:13:42	disruptek	i think it's more like the os allows select to unblock and tells it what, if any, fds are selected.
20:14:14	dom96	disruptek, that's how select normally works.
20:14:19	Zevv	shall we go practical, let's make a little nim snippet doing that and discuss how it works
20:14:43	disruptek	oddly, a -1 to select isn't working that way, right? due to something weird with nim's poll?
20:15:01	dom96	disruptek, it does, what gave you that idea?
20:15:20	rayman22201	I did not think that two different blocking primitives could interact in this way. I always assumed that the OS will only let you block on type of thing. You are saying that I can have a thread block on both a select and a semaphore, and get signals from both (with separate entry points back into my program)
20:15:23	disruptek	oh, i thought ray said a -1 was still unblocking.
20:15:57	rayman22201	no -1 blocks, the stdlib just doesn't use it by default :-)
20:16:07	disruptek	i'm not sure what these signals are you're speaking of, but i only just started paying attention.
20:16:13	disruptek	oh, okay.
20:16:53	rayman22201	I don't see how you can block on both a semaphore and a select at the same time.
20:17:07	Zevv	rayman22201: a poll (epoll,select,poll) system call returns in 3 cases: one of it's watched file descriptors has an event, the timeout occurs, or a the process receives a unix signal
20:17:23	Zevv	so, you *can not* block on a wait condition and on a poll at the same time
20:17:32	Zevv	that is the whole essence of the problem we are solving here
20:17:59	disruptek	technically, you could have one side block on the semaphore and one on the select; that's a deadlock. ;-)
20:17:59	Zevv	so there is a trick needed: the poll is doing its normal poll business, but it needs to be woken up when there is other work to do that it can not know of
20:18:02	dom96	Indeed, which is why I'm predicting that the process receives a unix signal when the `spawn`ed procedure finishes
20:18:04	rayman22201	exactly. This is why I don't understand how it is working in my sample program?
20:18:16	disruptek	can we see the sample?
20:18:32	dom96	Zevv, you seem to be implying that this trick needs to be implemented, it appears to already be in place
20:18:46	rayman22201	I think what @Dom96 is saying must be true. the semaphore must send a unix signal (or windows equivalent on windows)
20:18:57	rayman22201	hold on. I will share the sample
20:19:16	Zevv	part of my confusion: are we talking about the new synchronization mechinism that was just made a few days ago?
20:19:36	dom96	Zevv, which mechanism is that? Your PR?
20:19:52	rayman22201	http://ix.io/1OLc
20:20:08	rayman22201	I'm using the version of spawn that we created on the livestream
20:20:24	Zevv	rayman22201: that was the asyncFlowVar branch?
20:20:28	rayman22201	yes
20:20:40	Zevv	dom96: that one
20:22:04	disruptek	you spawn a thread and the other one is polling. are you saying the poll isn't running? is returning? what?
20:22:10	dom96	Yes, we are talking about that. Specifically: the only way that sample can work is if the FlowVar is signalled by the OS somehow.
20:22:31	dom96	(And the signalling happens in the same thread that is blocked on `poll`)
20:22:54	Zevv	that is the 'pipe' we are talking about
20:23:22	Zevv	the thread that reads from stdin writes into the pipe
20:23:30	Zevv	the other end of the pipe is watched in the poll set
20:23:37	dom96	Huh?
20:23:39	Zevv	so there is data availabe, poll wakes up and voila
20:23:47	dom96	Is this what you're proposing we implement or how threadpool currently works?
20:23:55	Zevv	that is how it now works
20:24:09	Zevv	and there is nothing wrong with that!
20:24:23	rayman22201	no it's not. The spawned thread never gets the other end of the pipe
20:24:30	Zevv	pfff,
20:24:31	Zevv	wait
20:24:32	dom96	I need evidence of this
20:24:44	dom96	Do you have links that show this "pipe" being added to the set of FDs watched by epoll?
20:24:45	Zevv	I will prepare, give me one minute
20:25:01	Zevv	wait
20:25:04	Zevv	i'm typing
20:25:59	rayman22201	"the other end of the pipe is watched by the poll set." where? There is no code for this.
20:26:05	Zevv	there *IS*
20:26:06	Zevv	wait
20:26:50	rayman22201	I can make another example with no IO
20:30:32	Zevv	http://ix.io/1OLg
20:30:39	Zevv	and there it is
20:32:23	Zevv	in the nim libs the path is newAsyncEvent -> newSelectEvent(creates eventfd) -> registerEvent(puts it in the epoll set) -> trigger(sends data)
20:33:11	dom96	This is code that rayman added I'm pretty sure
20:33:40	dom96	This doesn't answer my question :(
20:33:54	Zevv	this code has been there since jul 5 2016
20:34:03	dom96	can you show me the code?
20:34:38	Zevv	sure: lib/pure/ioselects/ioselectors_epoll.nim
20:34:45	Zevv	newSelectEvent(), trigger(), etc
20:35:14	dom96	That's the API
20:35:18	dom96	What calls these functions?
20:35:45	rayman22201	@Zevv the trigger gets called from the main thread
20:35:45	Zevv	that is the new rayman code, commit 7fac17e
20:35:51	dom96	precisely
20:36:08	dom96	This doesn't explain how `trigger` gets called
20:36:14	Zevv	no rayman, check the pid of my strace output where the write(3, ...) happens
20:36:29	Zevv	nimFlowVarSignal() does the trigger
20:36:37	rayman22201	right
20:36:58	Zevv	that used to only signal the waitcondition/semaphore
20:37:12	Zevv	but it now also triggers the pipe to the receiving thread
20:37:31	dom96	yes, what calls nimFlowVarSignal?
20:37:56	Zevv	magic
20:38:01	*	dom96 leaves
20:38:12	Zevv	really, its in compiler/
20:38:15	Zevv	so that's offlimits for mortals
20:39:06	Zevv	joking aside, does this all make sense rayman22201?
20:39:25	rayman22201	I found the magic. @Zevv is right: https://github.com/nim-lang/Nim/blob/9db369063c4d14d775015df8f7490d23603827f3/compiler/spawn.nim#L107
20:39:53	rayman22201	My faith in my understanding of Operating System scheduling is restored.
20:40:04	*	abm joined #nim
20:40:10	Zevv	dom96 is now gone, I guess we should proceed some other time
20:40:18	dom96	I'm not really gone :)
20:40:21	Zevv	:)
20:40:26	Zevv	so, dom96 also with me?
20:40:31	Zevv	does it all make sense still?
20:40:57	dom96	I'm frustrated.
20:41:21	Zevv	you'll get over it :)
20:41:33	dom96	Your explanation doesn't explain anything I didn't already know
20:41:43	Zevv	no, but for rayman22201 it did I hope
20:41:50	rayman22201	yes, it helped me
20:42:06	Zevv	ok, now back: what are we still confused about?
20:42:23	Zevv	because there was a lot of confusion going on all the time :)
20:42:30	rayman22201	I thought `nimFlowVarSignal` was called in the main thread (partially because Dom told me this was the case.) That is not true.
20:42:41	dom96	what
20:42:58	rayman22201	It may have been my misunderstanding of what you said.
20:43:08	dom96	No, it is called in the main thread
20:43:12	dom96	there is no other way it can work
20:43:14	rayman22201	It is not.
20:43:16	Zevv	it is not
20:43:19	Zevv	check the strace
20:43:32	Zevv	it is not supposed to - the main thread is doing the epoll()!
20:43:33	rayman22201	you don't need the trace, just look at the code I just linked
20:44:30	Zevv	the main thread does not know if the worker thread is done, it is autistically waiting for file descriptors and can do nothing else
20:44:49	Zevv	so the eventfd() file descriptor is just a way to call BOOO so it wakes up
20:46:15	disruptek	yes.
20:47:20	dom96	This shouldn't be possible
20:47:31	dom96	Because each thread has its own heap
20:47:36	dom96	The AsyncEvent is allocated on the main thread
20:47:36	*	jjido quit (Quit: My MacBook has gone to sleep. ZZZzzz…)
20:47:41	disruptek	life would be pretty difficult without the os helping us out here.
20:47:51	dom96	so how can the other thread access it?
20:47:57	dom96	That is what I based my assumptions on
20:48:04	disruptek	access what?
20:48:38	rayman22201	threads can access other threads heaps, it's just not safe.
20:49:15	rayman22201	we get lucky because there is no way the GC will collect the asyncEvent because the thread is stuck waiting on the read. It can't GC and wait on the select at the same time.
20:49:22	Zevv	yes, but the trigger is sent through a file descriptor, which is perfectly safe to do
20:49:41	FromGitter	<kaushalmodi> does anyone use the `comprehension` package?
20:49:42	dom96	yes, but a ref object stores the fd, which isn't safe
20:49:59	disruptek	it doesn't need to share it, though.
20:50:02	rayman22201	what @Dom96 is saying is that in order to know what fd to trigger on, it has to read the fd from the other thread.
20:50:02	dom96	but yes, you're likely right, we get lucky here and this works
20:50:11	FromGitter	<kaushalmodi> I am trying to translate this snippet from `lc` to `comprehension.comp`: https://play.nim-lang.org/#ix=1OLo
20:50:15	dom96	I expected the compiler to complain about this access
20:50:17	disruptek	no, they have identical information.
20:50:19	dom96	if it was cross-thread
20:50:27	disruptek	not that it matters.
20:50:28	rayman22201	The annoying part is that it is hard to see because of all the compiler magic going on. the `wrapSpawnProc` that does this is part of a freaking compiler rewrite pass
20:50:34	dom96	since it complains about GC-safety nearly constantly
20:50:56	rayman22201	all of that is bypassed because of the compiler magic... lol
20:51:17	rayman22201	https://github.com/nim-lang/Nim/blob/58e0dad3712d4e423fd26bd91b26b8a31a85bf33/compiler/ccgexprs.nim#L2250
20:52:02	Araq	told you to use yuri's threadpool
20:52:09	Araq	I dunno why you don't listen
20:52:15	Zevv	hehe
20:52:23	dom96	because you didn't explain why
20:52:25	dom96	you just said "use it"
20:52:32	Zevv	its not in the stdlib. And we can't just try out random nibmles and hope they are the right ones
20:53:30	rayman22201	especially when the whole point of this exercise is to make the stdlib better... (not that yuri is random. but still)
20:54:38	FromGitter	<awr1> has there been any talk of a work-stealing pool for nim
20:54:49	rayman22201	Yuri's pool does that
20:54:52	FromGitter	<awr1> or is that too hard b/c of the per-thread heap model
20:55:13	*	Vladar quit (Remote host closed the connection)
20:55:24	rayman22201	well, depends on how the data.
20:55:27	rayman22201	https://github.com/yglukhov/threadpools
20:55:38	FromGitter	<kayabaNerve> Errors? I'd like to complain about how IndexErrors aren't tracked by raises.
20:57:35	FromGitter	<kayabaNerve> Or really, I'd like to know why so I know not to complain.
20:58:07	Araq	read the spec, there are no IndexError
20:58:14	FromGitter	<awr1> i remember reading a paper a while back on implementing work-stealing using a deque structure and atomic CAS
20:58:32	Araq	you can't catch IndexErrors, instead fix your code
20:58:59	rayman22201	so, what is the conclusion here? @Zevv, @Dom96? Are we all on the same page about how flowVars are working in this model? What is the next step?
20:59:06	FromGitter	<kayabaNerve> I use the term complain a lot, and sure, they're complaints, but I believe they're purposeful
20:59:29	FromGitter	<kayabaNerve> Araq: You can.
20:59:44	FromGitter	<kayabaNerve> ... or at least I'm pretty sure I have before
20:59:53	FromGitter	<kayabaNerve> Let me check the playground. I'm on my phone right now
21:00:32	Zevv	afaik: the current setup works just fine, the pipe mechanism is proved technology. The only *but* is that it eats fds for breakfast, thus the other adventure with reusing the same fd for different event
21:01:01	rayman22201	Dom's trick will still work in this model.
21:03:30	FromGitter	<kayabaNerve> https://ibb.co/qYGhwvm
21:03:50	FromGitter	<kayabaNerve> You can catch IndexErrors thrown by invalid seq accesses.
21:04:30	FromGitter	<kayabaNerve> And that site doesn't work well on mobile at all
21:04:40	rayman22201	the big issue is that FlowVar is taking advantage of an unexpected interaction between select, the GC, and threads. It's only safe because the main thread is essentially frozen on the select, so we can read and write the memory from another thread safely (to get the fd). It's a big hack...
21:04:44	FromGitter	<kayabaNerve> Well. It works poorly
21:04:45	Araq	well you can't, why? because the spec says so.
21:04:54	FromGitter	<kayabaNerve> Kept deleting what I typed
21:05:01	Araq	the implementation doesn't follow the spec.
21:06:10	FromGitter	<kayabaNerve> So. IndexErrors are real things not tracked by raises yet catchable.
21:06:28	disruptek	rayman22201: why can't both parties simply agree on what the fd will be?
21:06:49	Araq	they are not catchable with -d:danger
21:07:14	dom96	rayman22201, yeah, this is problematic, I bet it hides some big problems
21:11:40	rayman22201	@dom96, we can just put a lock around asyncEvent. boom it's safe now :-)
21:12:16	dom96	I'm not so sure
21:12:21	dom96	the problem is that the other thread owns the memory
21:12:22	rayman22201	@disruptek, you can't know in advance what fd you will get. It's whatever the OS decides to give you.
21:12:29	dom96	it could deallocate it from under us
21:12:33	Zevv	it's global
21:12:42	Zevv	becasue it is in the async part, not in the selector
21:13:02	rayman22201	It will live as long as the asyncDispatcher lives. exactly
21:13:02	Zevv	and it exists before any other threads spawn, so putting a lock on it should be fine
21:13:11	Zevv	(which, i have been told, is in the book of famous last words)
21:13:41	rayman22201	lol
21:14:00	dom96	Well, I'll let Araq decide this
21:14:22	disruptek	it's not even an issue.
21:14:22	Zevv	but I must now leave this discussion - a decade ago I made a resolution never to use shared memory and mutexes ever again in my life because these make up a large fraction of things I regret in my career
21:15:18	Zevv	Araq will tell us to use some other threading lib - we might as well move to chronos while we're doing that and let Status do all maintainance for us
21:15:36	disruptek	sounds good to me.
21:16:00	rayman22201	@Araq is telling us to use Yuri's threading lib and Status is telling us to use Chronos for async.... what hope does the stdlib have :/
21:16:23	disruptek	this is kinda how a quality stdlib gets built.
21:16:25	*	shomodj joined #nim
21:16:37	Zevv	^^^
21:16:46	Araq	^ exactly.
21:17:18	rayman22201	Ok. so we should stop working on all of this and instead rip out the current implementations and replace them with these other implementations?
21:18:12	disruptek	only if you want experts to provide and support a quality contributions.
21:18:21	Araq	yeah. eventually. But I see this an experiment, you are learning how this thing needs to be setup
21:18:38	Araq	that's what I told you to not focus on the PR
21:18:45	Araq	*why
21:19:16	rayman22201	Ok. Well in that case. experiment complete. But I want to contribute to the community and right now I just feel like I'm spinning my wheels.
21:19:31	Araq	how so? how is it complete?
21:19:45	Araq	are we sure gcsafety is not violated?
21:19:46	disruptek	that's silly, you're one of a dozen people closest to understanding this critical piece of infra.
21:20:15	rayman22201	a piece of infra that you just said that you want removed.
21:20:28	Araq	look, Python's async implementation is probably it's 3rd implementation
21:20:48	disruptek	if chronos is a superior product, sure, it could be a good upgrade if it also meets the needs of nim's stdlib.
21:21:05	disruptek	but it's hard to know what it buys you when you don't even understand the problems it solves.
21:21:11	Araq	we don't throw the stdlib's async away. But at the same time it's not for forever either.
21:21:54	*	shomodj_ joined #nim
21:22:16	Araq	and the threadpool is simply in a poor shape, but even if it weren't, you complain in #nim how it's built with compiler magic
21:22:23	Araq	so that it is hard to understand.
21:22:39	Araq	which is correct, so use something more hackable for the experiment
21:23:02	rayman22201	the problem is, at that point, I'm making a PR for Yuri's threadpool, not the stdlib threadpool.
21:23:42	disruptek	and that's bad why? ;-)
21:23:50	Zevv	rayman22201: my conclusion is that we simply are on the right track. people had different assumptions about inner workings, causing a lot of confusion. THat is resolved, and we basically know the next step. Lets just make that implementation and see what starts smoking first when we push it hard
21:24:23	Araq	patch yuri's threadpool, submit it as a PR
21:24:31	Araq	to Nim's repo
21:24:51	Araq	write tests, no question will be asked and I'll throw away my code
21:24:55	*	shomodj quit (Ping timeout: 246 seconds)
21:24:59	Zevv	rayman22201: there is no problem here - put a lock over the thing and it should be just fine.
21:25:14	disruptek	but you won't need the lock. ;-)
21:25:17	*	absolutejam quit (Ping timeout: 268 seconds)
21:25:51	Zevv	right :) Anyway, I'm glad we are now all talking about the same thing, and I'm off for my daily downtime.
21:26:10	rayman22201	@Zevv, thanks again. enjoy your downtime
21:26:15	Araq	so once more: threadpool can be replaced with yuri's implementation
21:26:18	*	Zevv poll(NULL, 8*60*60)
21:26:36	rayman22201	@Araq that seems like the PR I should be doing
21:26:39	rayman22201	apparently
21:26:45	Araq	but we won't throw away asnc as easily
21:27:09	Araq	async is battle-tested, yeah it bugs, it's a complexe piece of infrastructure
21:27:11	rayman22201	by making stdlib async work with spawn we are making it more likely for people to use spawn, so we need a good spawn.
21:27:50	rayman22201	I've just been forever frustrated by the stdlib async vs. chronos async split for a long time
21:28:14	*	absolutejam joined #nim
21:28:31	rayman22201	I don't really care who wins. they are both performant, and both battle tested....
21:28:55	rayman22201	but for the end user, it's just confusing, and annoying.
21:29:18	Araq	well currently chronos is winning, it supports 'cancel', was built after the stdlib's async
21:29:58	Araq	so it learned lots of lessons.
21:30:29	rayman22201	chronos also already has working async locks and async streams (and soon channels apparently). So it's more feature complete.
21:30:53	Araq	it's like complaining about too much candy, man
21:31:01	rayman22201	It makes me think, why waste time with the stdlib async?
21:31:55	*	solitudesf quit (Ping timeout: 246 seconds)
21:32:04	Araq	ok, how about this: what do you think we should do?
21:32:04	rayman22201	but less experienced programmers will pick the stdlib async, and have a worse experience b/c they will use what is included....
21:32:28	rayman22201	Chronos should be the stdlib async.
21:32:36	Araq	you know the context, the mystical voices whispering 'v1'
21:34:30	Araq	today we released the 2nd release candidate for v1. What to do next? throw away async. Doesn't look like a plan to me. :-)
21:36:12	dom96	rayman22201, imagine how I feel about the chronos split
21:36:26	rayman22201	I understand the constraint here. I just think it looks bad that 'v1' is going to ship with an obviously inferior set of features...
21:37:25	dom96	If you want to help, please port these features to the stdlib async
21:37:28	Araq	well "obviously inferior". I never used chronos and I did use async (believe it or not)
21:37:47	dom96	"obviously inferior" sounds pretty harsh
21:38:13	rayman22201	ok, particularly in the case of async vs. chronos that is a bit harsh. They are neck and neck really.
21:39:05	dom96	They probably aren't, but one is being developed by a set of employees being paid a full-time wage to work on it
21:39:13	Araq	let me say it differently: replacing spawn by a better or even a simpler one is welcome and could make it into v1
21:39:14	rayman22201	I don't know if "port these features to async" is the right answer either. We will always being playing catchup that way.
21:39:25	rayman22201	exactly @Dom96, it makes more sense to take advantage of that
21:39:41	Araq	and making async work with the threadpool is obviously very useful
21:40:03	Araq	and we cannot abandon async either. So you're helping Nim's core.
21:40:24	dom96	Sorry but this simply isn't going to happen for v1, it's too much breakage
21:40:37	*	ng0 joined #nim
21:40:44	rayman22201	I did not mean to assume v1.
21:41:02	rayman22201	but my understanding is that there is no roadmap to this at all, even for v2.
21:41:14	rayman22201	for v1, it's too late. I get that.
21:41:30	rayman22201	and I hear you loud and clear about threadpool @Araq.
21:42:29	dom96	I would prefer we revamp async completely for v2
21:42:54	dom96	In particular to make it heap-free
21:43:04	dom96	no idea if that's even possible of course, but it's worth a try
21:43:27	dom96	chronos is so similar to stdlib async that it's a shame to switch to it and deal with the random breakages that were introduced
21:43:38	rayman22201	I strongly believe that we should adopt chronos for v2. They are paying a team to work on that full time. How can we ever compete with that?
21:43:47	rayman22201	or why would we want to?
21:44:21	dom96	Just because they are paid now doesn't mean they will be forever
21:44:41	Araq	sure, but it's open source
21:47:56	dom96	Well, I'll admit, I'm very biased here so you'll have a super hard time convincing me.
21:48:17	dom96	1.0 isn't even out yet, let's see what happens
21:48:20	Araq	but you're right, why would we want to compete with them. We should maintain what we have to keep up the trust people have in Nim. I can write a better strutils tomorrow
21:48:44	Araq	now what? should we remove strutils from Nim and replace it with the "better" version?
21:49:11	rayman22201	That is essentially what you do today?
21:49:23	Araq	what?
21:49:24	rayman22201	see all the regex libs in the stdlib
21:49:36	disruptek	those are additative.
21:49:40	Araq	well the number is two.
21:49:49	Araq	there is also a wrapper
21:49:57	rayman22201	they aren't really additive, they each have different feature sets
21:49:58	Araq	the wrapper doesn't count
21:50:25	disruptek	the point is, none of them got taken out behind the woodshed and tol' to put their head down.
21:50:25	dom96	btw, I have absolutely no idea why you'd ever need async locks
21:50:46	Araq	well I wrote re.nim, people complained, contributed nre, we decided to adopt nre and deprecate re
21:50:52	rayman22201	you need it for when you have threads mixed with your async @dom96 :-)
21:50:59	Araq	then we figured we like re.nim better and now we have both
21:51:08	rayman22201	instead of blocking on a lock you can do another async task.
21:51:30	Araq	and also all my code uses re.nim and I am not happy to rewrite it
21:51:43	disruptek	and why should you be?
21:52:04	Araq	same with async, you think everybody is happy to constantly update their code.
21:52:09	disruptek	software is hard enough to write once, why do we need to write it twice?
21:52:32	dom96	rayman22201, true, I'm too much in the heap-per-thread mindset now. Nim changed me :P
21:53:00	Araq	but that's not true, it's tiresome to do this. especially when the development process is longer than a decade
21:53:08	rayman22201	but this is the argument for adopting Chronos? why rewrite all those features for the stdlib async when you can just use the already written versions?
21:53:14	dom96	I still think that is a good threading model, especially for servers which typically serve thousands of requests with no state shared between requests
21:53:27	dom96	(the state can just be looked up in a DB)
21:54:01	dom96	btw regarding regex, we can take another example, there is a Nimble package which implements native regex in Nim
21:54:09	disruptek	the argument for adopting chronos is to put those closest to the solution and most heavily invested in it, in charge of the implementation.
21:54:11	rayman22201	lol. sure. you are passing the problem to the DB. (DB's are really good at this though.)
21:54:12	dom96	Should we just replace the current stdlib regex modules with that package?
21:54:15	dom96	I don't think so
21:54:50	disruptek	don't conflate replace with adopt.
21:55:32	Araq	for v2 it's a perfectly fine outcome to say "async is in maintainance mode, old code keeps working, new code should seriously consider chronos"
21:57:04	rayman22201	That's practical. I respect that decision.
21:57:38	Araq	it's one possible outcome, I haven't QA'ed chronos btw
21:58:22	rayman22201	If that happens, It makes me (and maybe others) not want to contribute to stdlib async though. Just contribute to chronos instead...
21:58:37	disruptek	try to imagine a world in which you're glad that you can rely on a package X being present and working in your stdlib. that doesn't happen overnight, but it has to start somewhere.
21:59:20	Araq	look at Python, its stdlib has plenty of packages that were crappy compared to what Python's community came up with
21:59:40	rayman22201	@disruptek I agree with you. that is why I am arguing the way I am. I am trying to make the async I want to see in the world. lol
22:00:08	Araq	that's simply how it works. Yet Python is better off with keeping its stdlib
22:00:10	AlexMax	https://paste.ee/p/nvAT4
22:00:17	AlexMax	Is this an intended output of finish.exe?
22:00:22	rayman22201	@Araq, you are correct. In a way it's one of the down sides of the "batteries included" approach to a std lib.
22:00:54	*	elrood quit (Remote host closed the connection)
22:01:12	rayman22201	AlexMax: no it is not. You may have missed a step? do you have mingw installed?
22:01:23	Araq	AlexMax, is that with 0.20.2?
22:01:43	dom96	rayman22201, the async in the stdlib is pretty freaking awesome, please don't be discouraged from contributing to it
22:03:08	AlexMax	Yes
22:03:13	AlexMax	Yes that's with 0.20.2
22:03:29	dom96	rayman22201, also, by now I bet it has features which chronos doesn't have
22:03:33	Araq	at least that's not a regression
22:03:52	Araq	AlexMax, do you have C:\Program Files\mingw-w64 but not C:\Program Files\mingw-w64\bin ?
22:03:54	dom96	rayman22201, the biggest of which: it's supported by practically all nimble packages
22:04:00	AlexMax	I do have a directory called mingw-w64 but it is empty
22:04:14	Araq	ok, well remove this directory
22:04:14	AlexMax	possibly from uninstalling mingw-w64
22:04:21	rayman22201	lol. @dom96, that is a very good argument for the stdlib async. I give you that.
22:04:22	Araq	bug will be fixed in the next release
22:04:48	AlexMax	okay, working now, just thought it was worth bringing up
22:04:55	Araq	(we really need a expandFilename that doesn't raise exceptions ...)
22:05:06	AlexMax	would you like me to create an issue or do you have it?
22:05:23	Araq	an issue is nicer
22:05:39	AlexMax	10-4
22:05:41	Araq	but I already fixed it, but now I have a ticket number
22:06:47	Araq	dom96, rayman22201 maybe most packages can work with both chronos and the stdlib
22:06:57	Araq	I don't know how incompatible it is
22:07:12	Araq	I mean, .async + await, how can you improve on that DSL?
22:07:21	rayman22201	The biggest thing is that they decided on a fairly different api for their Future type...
22:07:31	dom96	they seemed to have changed most of the API
22:08:17	FromGitter	<kayabaNerve> Araq: Sorry, I didn't get your messages. I thought I sent my image and all other messages without any in between. Thanks for the info. Have any documentation for the danger define?
22:08:31	rayman22201	it's annoying. The big ideas and patterns are similar, but the chronos api is different enough to make it a PITA
22:09:05	disruptek	if you don't remove existing async, nothing breaks. what's the problem?
22:09:39	Araq	disruptek, package A uses stdlib, package B uses chronos and you want to use both in your program
22:10:09	disruptek	is there a symbol collision?
22:10:44	dom96	there is an event loop collision
22:10:52	dom96	it's less efficient
22:11:04	dom96	and you have to run two event loops at the same time and make sure they cooperate somehow
22:11:54	disruptek	hmm, there's your contribution, rayman22201 -- plugable async. ;-)
22:11:57	AlexMax	Araq: Done, #11770
22:14:29	FromGitter	<mratsim> @awr1 if you want an efficient work-stealing threadpool, each thread should have a deque to steal from.
22:14:40	FromGitter	<mratsim> A global deque doesn't work
22:15:10	FromGitter	<mratsim> also threadpools with a single queue will just move the contention to the lock protecting that queue
22:15:31	FromGitter	<mratsim> iirc, libdispatch (from Apple Grand Central Dispatch) randomly picks the queue to add the task too
22:15:47	FromGitter	<awr1> yeah that was mentioned in the paper
22:15:52	FromGitter	<mratsim> or maybe it was Cilk
22:16:12	Araq	mratsim: we could never reproduce these contention points
22:16:12	FromGitter	<mratsim> you can also pick the one with the less tasks
22:16:32	FromGitter	<awr1> i remember going to a talk by one of the cilk devs at intel ages ago
22:16:45	FromGitter	<awr1> which is how i found out about workstealing
22:16:46	Araq	it always was in the noise. but ok, our benchmarking probably sucked
22:17:08	Araq	but yeah, everybody else uses workstealing, it has won
22:17:46	disruptek	i think the issue is that it's hard to reproduce some of those troublesome workloads, but that doesn't mean you don't have to design around them.
22:17:49	FromGitter	<mratsim> Well, I'm looking into implementing some work-stealing threadpool for high performance computing, because OpenMP has some critical limitations that I will hit in a couple of months
22:18:10	*	abm quit (Ping timeout: 248 seconds)
22:18:47	dom96	btw I implemented prometheus recently with a nice async collector which allows you to see exactly which futures are pending in your program. It's an awesome way to diagnose async problems.
22:21:26	FromGitter	<mratsim> @awr1, I think this video is a nice recap on some of the challenges of a threadpool: https://youtu.be/zULU6Hhp42w?t=1695
22:25:46	FromGitter	<awr1> https://blog.molecular-matters.com/2015/08/24/job-system-2-0-lock-free-work-stealing-part-1-basics/
22:25:53	FromGitter	<awr1> i also remember reading this
22:25:56	*	darithorn joined #nim
22:27:23	FromGitter	<awr1> in a threadpool engine you can use a "global queue" but yeah like you say i'd hesitate to call it work stealing at that point, seems to me like it would cause too much contention
22:31:56	FromGitter	<awr1> i vaugely remember (and correct me if i'm wrong) in the deque structure you need to have one end for non-associated threads to "steal from" (as where are where incoming stolen jobs are pushed to), and that the other end is for the thread associated with the deque to pop and execute from
22:33:34	FromGitter	<awr1> and supposedly you can do this with just CAS and without heavier locks
22:34:41	disruptek	you don't need strict deques.
22:35:05	Araq	implement the "disruptor"
22:35:07	disruptek	gah, i cannot make nimteropt work at all.
22:35:28	rayman22201	@awr1 that looks really fun to implement :-)
22:43:56	FromGitter	<awr1> @disruptek you can use normal queues but i believe you would need to guard every queue with a lock
22:47:18	FromGitter	<mratsim> yes the thread takes task from one end and the thieves from the other
22:49:18	FromGitter	<mratsim> Then there are other considerations depending on the domain. For example, in high-performance computing, you want last in first out because the last in probably is depending on data hot in cache so it would be faster to process it, you wouldn't have to wait on memory
22:49:56	FromGitter	<mratsim> but in other cases you want fairer scheduling/optimize waits
22:50:08	*	darithorn left #nim ("Leaving")
22:52:49	*	ng0 quit (Quit: Alexa, when is the end of world?)
23:06:09	shashlick	@disruptek what are you trying
23:12:02	*	yeetcannon quit (Ping timeout: 248 seconds)
23:12:55	*	yeetcannon joined #nim
23:13:32	*	yeetcannon quit (Remote host closed the connection)
23:18:22	*	yeetcannon joined #nim
23:23:28	*	dddddd quit (Remote host closed the connection)
23:23:34	*	absolutejam quit (Ping timeout: 258 seconds)
23:38:51	*	shomodj_ quit (Quit: My MacBook has gone to sleep. ZZZzzz…)
23:49:08	*	krux02_ joined #nim
23:51:35	*	krux02 quit (Ping timeout: 250 seconds)
23:55:43	*	smitop joined #nim