00:34:55 | * | ormiret quit (Ping timeout: 268 seconds) |
00:35:02 | * | tk quit (Quit: Well, this is unexpected.) |
00:35:20 | * | msirabella joined #nim |
00:36:11 | * | mjsir911 quit (Ping timeout: 268 seconds) |
00:36:20 | * | tk joined #nim |
00:36:47 | * | Yardanico quit (Ping timeout: 268 seconds) |
00:36:49 | * | ormiret joined #nim |
00:36:54 | * | Yardanico_ joined #nim |
00:37:35 | NimEventer | New post on r/nim by OptimizedGarbage: Experience with Dr. Nim or other correctness-assurance techniques?, see https://reddit.com/r/nim/comments/s6js59/experience_with_dr_nim_or_other/ |
00:40:08 | FromDiscord | <Rika> if you're writing a game you dont need formal proving id say...... |
00:40:36 | FromDiscord | <Rika> though it certainly isnt a "do not" |
00:40:46 | FromDiscord | <Rika> just kinda a big time consumer |
00:47:51 | * | krux02_ quit (Remote host closed the connection) |
00:57:40 | FromDiscord | <Patitotective> In reply to @auxym "The wrapped library itself": 👍 |
01:11:52 | FromDiscord | <Patitotective> any clue? |
01:11:52 | FromDiscord | <Patitotective> sent a code paste, see https://play.nim-lang.org/#ix=3MCN |
01:12:35 | FromDiscord | <Elegantbeef> There is no `wxnim` module |
01:12:42 | FromDiscord | <Elegantbeef> you want `import wxnim/wx` |
01:14:09 | FromDiscord | <Patitotective> Oh, thanks 🙃 |
01:15:59 | * | NimEventer quit (Remote host closed the connection) |
01:16:41 | * | NimEventer joined #nim |
01:24:19 | FromDiscord | <congusbongus> I think if you use VSCode with nim extension it can autocomplete imports |
01:25:53 | FromDiscord | <Rika> nope |
02:06:18 | * | tk quit (Quit: Well, this is unexpected.) |
02:06:31 | * | tk joined #nim |
02:14:38 | FromDiscord | <Patitotective> sent a code paste, see https://play.nim-lang.org/#ix=3MCW |
02:15:39 | FromDiscord | <Elegantbeef> https://github.com/beef331/slicerator/blob/master/src/slicerator.nim#L37-L40 |
02:16:27 | FromDiscord | <Patitotective> 👍 |
02:24:19 | * | neurocyte0917090 joined #nim |
02:26:02 | * | neurocyte091709 quit (Ping timeout: 240 seconds) |
02:26:02 | * | neurocyte0917090 is now known as neurocyte091709 |
02:33:24 | * | noeontheend quit (Ping timeout: 250 seconds) |
02:37:43 | * | arkurious quit (Quit: Leaving) |
02:53:35 | * | noeontheend joined #nim |
03:00:12 | * | ejjfunky joined #nim |
03:45:59 | * | Gustavo6046 quit (Read error: Connection reset by peer) |
03:47:18 | * | Gustavo6046 joined #nim |
03:51:48 | * | Gustavo6046 quit (Read error: Connection reset by peer) |
03:53:09 | * | Gustavo6046 joined #nim |
04:21:56 | FromDiscord | <huantian> oh that's probably better than .reversed huh↵unless one day reversed is lazy |
04:22:10 | * | rockcavera quit (Remote host closed the connection) |
04:22:24 | FromDiscord | <Elegantbeef> It'll never be since it returns a sequence |
04:36:56 | * | Figworm quit (Remote host closed the connection) |
04:38:07 | * | Figworm joined #nim |
04:47:02 | * | noeontheend quit (Ping timeout: 240 seconds) |
05:23:51 | FromDiscord | <evoalg> slicerator ftw! |
05:25:03 | FromDiscord | <Elegantbeef> 100% of my users say the same, though you're 100% of my users 😛 |
05:38:24 | FromDiscord | <Rika> I’d probably use it IF I WERE PROGRAMMING |
06:06:01 | * | ejjfunky quit (Ping timeout: 256 seconds) |
06:23:42 | FromDiscord | <Elegantbeef> Shame would be a lot cooler if you were |
06:28:19 | * | neocron joined #nim |
06:30:32 | * | Doraemon quit (Ping timeout: 240 seconds) |
06:50:57 | FromDiscord | <Rika> Aw shucks too bad I’m wasting my time |
07:26:01 | * | PMunch joined #nim |
08:05:15 | arkanoid | I really think arraymancer/datamancer should switch it's underlying memory model to apache arrow |
08:05:53 | arkanoid | All mayor data manipulation players are going in that direction |
08:13:38 | FromDiscord | <that_dude> What is it using right now, and how can I read about the differences? |
08:14:09 | * | Lord_Nightmare joined #nim |
08:15:53 | * | Figworm quit (Remote host closed the connection) |
08:28:04 | arkanoid | that_dude, the apache arrow model is described in it's own site. From what I know, the "numerical" based arrays are 1:1 between ArrayMancer <> Numpy <> Arrow, but strings are different among the three. Pandas now has a new "string[arrow]" type to use arrow memory model for strings, but Arraymancer doesn't do that and works with pointers |
08:28:40 | arkanoid | this piece of code works with --gc:refc but doesn't compile with --gc:arc, can you tell me how to fix? https://play.nim-lang.org/#ix=3ME7 |
08:29:00 | FromDiscord | <Rika> Maybe worth pinging mratsim |
08:29:52 | arkanoid | Rika: https://play.nim-lang.org/#ix=3ME7 |
08:29:54 | arkanoid | ups |
08:29:59 | arkanoid | Rika: https://github.com/mratsim/Arraymancer/issues/417#issuecomment-593155129 |
08:31:45 | FromDiscord | <Rika> Ah I see |
08:31:59 | FromDiscord | <Rika> I’m not math enough to get into it |
08:33:40 | arkanoid | polars (rust), cudf (nvidia), pandas (python) .... https://arrow.apache.org/overview/ |
08:45:02 | * | Figworm joined #nim |
08:46:00 | * | jjido joined #nim |
08:50:57 | NimEventer | New thread by Giaco: Setting finalizer on object hierarchy won't compile with --gc:arc, see https://forum.nim-lang.org/t/8820 |
08:54:50 | * | Gustavo6046 quit (Quit: Leaving) |
09:04:50 | FromDiscord | <mratsim> In reply to @arkanoid "I really think arraymancer/datamancer": The standard data layout is DLPack: https://github.com/dmlc/dlpack |
09:05:05 | FromDiscord | <mratsim> Arraymancer layout is compatible with Numpy, PyTorch and Tensorflow |
09:06:11 | arkanoid | mratsim, my experiments shows that that is true only for numerical data, but not string-based one, correct? |
09:07:13 | FromDiscord | <mratsim> With the Laser backend change at least at a low-level supports passing buffer in zero-copy mode between Arraymancer and others framework, to import a buffer you just need to set memowner to false: https://github.com/mratsim/Arraymancer/blob/master/src/arraymancer/laser/tensor/datatypes.nim#L38-L40 |
09:07:47 | FromDiscord | <mratsim> In reply to @arkanoid "<@570268431522201601>, my experiments shows": It's not possible for strings. Memory representation of strings in Nim is different from C or Python |
09:09:44 | arkanoid | but wouldn't be possible to adopt arrow memory model for strings in Arraymancer? Numpy can do that: https://arrow.apache.org/docs/python/numpy.html |
09:11:08 | arkanoid | currently, working with strings dataset locks nim datascience experience into limited set of tools |
09:11:28 | FromDiscord | <mratsim> Sure, but there are very few algorithms that makes sense for strings in Arraymancer. It's more for Datamancer. |
09:12:27 | arkanoid | sure, I'm talking about Arraymancer as Tensor for Datamancer |
09:14:24 | arkanoid | I've already succeeded in zero-copy passing from arraymancer -> numpy -> arrow (looking for link right now) but that's just for numerical data. When I work with datasets, I always have some string labels somewhere |
09:14:31 | FromDiscord | <mratsim> I think the best would be to create a package for the core memory representation, Call it "I took an arrow in the nim" and Have Arraymancer only have the algorithm. |
09:16:00 | arkanoid | here https://github.com/SciNim/scinim/issues/8#issuecomment-991844883 |
09:16:02 | FromDiscord | <Rika> LOL |
09:16:47 | FromDiscord | <mratsim> Either you wrap C/C++ like we did with flambeau https://github.com/SciNim/flambeau/blob/master/flambeau/raw/bindings/rawtensors.nim↵↵Or since it seems like Arrow is a versioned spec, you implement the spec from scratch. |
09:18:27 | arkanoid | well, there's https://arrow.apache.org/docs/format/CDataInterface.html, then arrow exposes C api via gobject and I've successfully wrapped it automatically with futhark https://github.com/PMunch/futhark/issues/6#issuecomment-986969310, but there's also manual wrapping https://nimble.directory/search?query=arrow |
09:21:04 | FromDiscord | <mratsim> It seems like a simple enough format to implement in a separate lib from scratch |
09:21:14 | FromDiscord | <mratsim> the spec goes into physical layout details like https://arrow.apache.org/docs/format/Columnar.html#fixed-size-list-layout |
09:21:31 | FromDiscord | <mratsim> We just need a library name cc @Vindaar @Clonkk . |
09:21:50 | FromDiscord | <mratsim> and time. |
09:22:25 | FromDiscord | <Rika> ~~Skynim~~ |
09:22:33 | FromDiscord | <mratsim> lol |
09:22:36 | FromDiscord | <mratsim> Archer |
09:22:45 | FromDiscord | <mratsim> or if there is a "Archer king" we pick that name |
09:23:02 | FromDiscord | <Elegantbeef> Sterling |
09:23:18 | FromDiscord | <Elegantbeef> Someone get the sterling reference please |
09:23:30 | FromDiscord | <amadan> phrasing |
09:23:40 | FromDiscord | <amadan> or do we not do that anymore? |
09:23:41 | FromDiscord | <Rika> Apollo? It’s a common name of course |
09:23:42 | FromDiscord | <Elegantbeef> Ehhh |
09:24:02 | FromDiscord | <Rika> Oh sweet “Apollo stays the serpent Python” |
09:24:03 | FromDiscord | <Rika> Lmfao |
09:24:14 | FromDiscord | <Rika> Slays |
09:24:16 | FromDiscord | <Clonkk> A native implementation of the Arrow specs would be dope↵(@mratsim) |
09:24:19 | FromDiscord | <Rika> I typed stays and I don’t know why |
09:24:35 | FromDiscord | <Elegantbeef> Just call it fletching cause it gets scinim where they want to go |
09:24:35 | arkanoid | Narrow |
09:25:15 | FromDiscord | <Elegantbeef> Damn ark with the genius "add the first letter of Nim to the word" 😛 |
09:25:43 | PMunch | Narkanoid |
09:25:49 | FromDiscord | <Clonkk> Quiver ? Fleche (to follow Flambeau footstep of french wordplay) ?↵(<@709044657232936960_arkanoid=5b=49=52=43=5d>) |
09:25:59 | FromDiscord | <Rika> Narco xd |
09:26:02 | FromDiscord | <Elegantbeef> He's not a nark |
09:26:10 | FromDiscord | <Rika> Quiver is a common name |
09:26:19 | FromDiscord | <Rika> I believe Apache has something named that |
09:26:37 | FromDiscord | <Clonkk> Go full meta then and call it `U+02192` |
09:26:51 | FromDiscord | <Clonkk> (unicode for right arrow) |
09:26:52 | arkanoid | reminds me Narc knight https://www.youtube.com/watch?v=CEdv1VEaogA |
09:27:16 | FromDiscord | <Rika> In reply to @Clonkk "Go full meta then": Very catchy name |
09:27:31 | FromDiscord | <Rika> Hey do you use that library with numbers for a name |
09:27:34 | FromDiscord | <Clonkk> Simple to remember as well |
09:27:40 | PMunch | Haha :P |
09:28:35 | FromDiscord | <Rika> Lul Arrowmancer |
09:28:50 | FromDiscord | <Clonkk> rofl |
09:29:08 | FromDiscord | <Clonkk> Isn't it a bit old to add "-mancer" to everything \:D |
09:29:09 | PMunch | Hmm, I'm generalising the AVR stuff from Badger. I want to create a framework/library for this, but I'm struggling with a name |
09:29:25 | FromDiscord | <Rika> I mean wasn’t it supposed to integrate with Arraymancer |
09:29:28 | PMunch | I could call the framework Badger, and the keyboard firmware Badger-board or something |
09:30:58 | FromDiscord | <evoalg> I thought of a name but realized it was too cupid to mention |
09:31:55 | FromDiscord | <Clonkk> Call it `dart` like a small arrow, I'm sure noone has ever done that |
09:32:02 | FromDiscord | <Clonkk> No confusion possible here |
09:32:36 | arkanoid | PMunch: you can always go get some inspiration from old flash times https://www.youtube.com/watch?v=hGlyFc79BUE |
09:32:42 | FromDiscord | <mratsim> Or bolt |
09:33:31 | PMunch | arkanoid, the framework is called badger badger badger badger badger badger badger badger badger badger |
09:33:40 | PMunch | And the keyboard is called mushroom |
09:33:41 | arkanoid | make sense |
09:34:04 | arkanoid | and let me guess python api |
09:34:30 | FromDiscord | <evoalg> @mratsim you said "If you don't use sequences, strings and ref types, there is no garbage to collect" ... so if I only use arrays and such, could I use --gc=none? (remember you're talking to a baby who doesn't understand a lot) |
09:34:48 | FromDiscord | <Elegantbeef> Yes |
09:34:55 | FromDiscord | <Elegantbeef> I guess exceptions aswell |
09:35:08 | FromDiscord | <evoalg> would that make it possible faster? |
09:35:20 | FromDiscord | <Elegantbeef> Allocations generally arent the reason programs are slow |
09:35:23 | arkanoid | also closures I guess, but unsure |
09:35:24 | FromDiscord | <Elegantbeef> Dumb allocations are |
09:36:00 | FromDiscord | <evoalg> so gc doesn't really slow things down much huh? |
09:36:30 | FromDiscord | <Elegantbeef> `ref`, `seq`, `closure`, `string` exceptions, are all the things that are heap allocated and create garbage, i'm sure i'm missing stuff |
09:36:45 | FromDiscord | <evoalg> ahhhhhh |
09:36:52 | FromDiscord | <Elegantbeef> Well it can slow down but generally the issue isnt the GC as much as not reusing memory or designing the code intelligently |
09:37:08 | FromDiscord | <Elegantbeef> Aside from the allocation a seq and an array are going to be the same speed to iterate over |
09:37:34 | FromDiscord | <evoalg> oh! |
09:38:44 | FromDiscord | <Elegantbeef> The issue with reference over value type is generally do to the cache locality, `ref` throws values all over the heap so when you're iterating over them you have to go back to ram more often |
09:38:51 | FromDiscord | <Elegantbeef> And going back to ram vs. cache is slooooow |
09:38:56 | PMunch | The greatest speed increase is definitely in designing your program and algorithms |
09:39:16 | FromDiscord | <mratsim> In reply to @PMunch "The greatest speed increase": Nope, it's when you do no work 😉 |
09:39:38 | PMunch | Well if you're not doing work you're not getting anything done, so speed is 0 |
09:39:43 | FromDiscord | <mratsim> no work > carefully designed work > accidentally quadratic |
09:39:43 | FromDiscord | <Elegantbeef> You're right, make all programs happen at compile time and cache every possible state! |
09:40:09 | FromDiscord | <mratsim> sometimes the only thing you can do is waiting (for the users, for the network, ...) |
09:40:22 | FromDiscord | <mratsim> async/await makes progress by waiting |
09:40:39 | FromDiscord | <mratsim> that's why you can handle millions of async tasks on a single CPU |
09:41:16 | FromDiscord | <Elegantbeef> "I dont get it my game's FPS is so low"↵"Have you tried giving inputs more often"😛 |
09:41:20 | PMunch | Yeah async is really neat if your code spends most of its time sitting around waiting for hardware |
09:41:46 | PMunch | Hmm, that could be an interesting game concept |
09:42:14 | PMunch | The faster you type the higher the framerate, and typing is somehow the core input |
09:42:33 | PMunch | I guess it would be similar to SuperHot though |
09:42:58 | FromDiscord | <Elegantbeef> Superhot meets typing of thedead |
09:43:16 | PMunch | Typing of the Dead? |
09:43:34 | FromDiscord | <Elegantbeef> https://en.wikipedia.org/wiki/The_Typing_of_the_Dead |
09:43:49 | FromDiscord | <Elegantbeef> Arcade shooter turned into keyboard instruction |
09:44:09 | FromDiscord | <Clonkk> I propose a remake of Speed about a programmer who has a bomb in his computer and if he stops typing below a certain speed the bomb explode |
09:44:11 | FromDiscord | <mratsim> In reply to @evoalg "<@!570268431522201601> you said "If": it depends on what you want to do, if you want to avoid having to use NimMain when you use Nim as DLL, yes avoid those types and also top-level variable (i.e. wrap everything in a function) |
09:44:35 | FromDiscord | <Elegantbeef> Yea that's going right over Evo's head |
09:44:44 | FromDiscord | <evoalg> that's right 😉 |
09:44:57 | FromDiscord | <evoalg> I realized I shouldn't have asked the question |
09:44:58 | FromDiscord | <mratsim> so what do you want to do? |
09:45:04 | FromDiscord | <Elegantbeef> Evo is new to system programming in general |
09:45:10 | FromDiscord | <mratsim> meh, only way to learn is by asking questions |
09:45:16 | FromDiscord | <mratsim> to Google at least |
09:45:38 | FromDiscord | <Elegantbeef> Evo just wants to not write unoptimized code that he learned to write from python, but he should ask a profiler instead of sages 😛 |
09:45:59 | FromDiscord | <mratsim> oh, don't worry about GC then? |
09:46:04 | FromDiscord | <mratsim> (edit) "then?" => "then." |
09:46:23 | FromDiscord | <Clonkk> Yeah the rules is usually "if you don't understand GC, then don't worry about GC" |
09:46:25 | FromDiscord | <evoalg> ok ... I won't worry 🙂 |
09:47:16 | FromDiscord | <mratsim> GC is for squeezing the last 10%. |
09:47:17 | FromDiscord | <Elegantbeef> My rule is "If it's slow profile it, if it's not carry on until it is" 😛 |
09:47:42 | FromDiscord | <Elegantbeef> GC's get a bad wrap but they're nice generally |
09:47:51 | FromDiscord | <Clonkk> So... Make it work, then make it fast, then make it work after breaking everything to make it fast \:D ? |
09:48:02 | FromDiscord | <mratsim> basically, avoid quadratic behavior and then you're in good shape. |
09:48:04 | FromDiscord | <Elegantbeef> Yes |
09:48:45 | FromDiscord | <Elegantbeef> You left out "avoid sequtils for iterating"↵(@mratsim) |
09:49:08 | FromDiscord | <mratsim> When I need iteration, I write my own for loops 😉 |
09:49:24 | FromDiscord | <evoalg> or use slicerator ftw! |
09:49:46 | FromDiscord | <mratsim> Or https://github.com/numforge/loop-fusion |
09:49:53 | FromDiscord | <Elegantbeef> There we go 😛 |
09:50:12 | FromDiscord | <Elegantbeef> Fuck that's hard to read |
09:50:19 | FromDiscord | <Elegantbeef> ` forZip i, x in var a, y in b, z in c:` ah yes |
09:50:50 | FromDiscord | <Elegantbeef> I have my new dumb `chain` macro which probably isnt much better `for i, x in chain a.items.filter(i > a.len div 2).map(x 10):` but alas |
09:50:56 | FromDiscord | <mratsim> or zero-copy views: https://github.com/mratsim/constantine/blob/master/research/kzg_poly_commit/strided_views.nim#L219-L245 |
09:51:24 | FromDiscord | <Elegantbeef> Depending what we're getting views into ... just use `[]` iterators |
09:51:56 | FromDiscord | <mratsim> I need to reverse, split or take 1 out of 2 very big integers. |
09:52:22 | FromDiscord | <Elegantbeef> Yea i just make toys so i dont need anything fancy |
09:52:51 | FromDiscord | <mratsim> that' the fancy part: https://github.com/mratsim/constantine/blob/master/research/kzg_poly_commit/strided_views.nim#L29 |
09:52:56 | FromDiscord | <mratsim> (edit) "that'" => "that's" |
09:53:06 | FromDiscord | <mratsim> `lent UncheckedArray[T]` |
09:53:07 | FromDiscord | <Elegantbeef> I recall talking about it prior |
09:54:24 | * | jjido quit (Quit: My MacBook Air has gone to sleep. ZZZzzz…) |
09:56:52 | FromDiscord | <evoalg> yay I can capture ctrl-c ... https://stackoverflow.com/questions/54713213/how-do-i-handle-ctrl-c-interrupt-in-nim |
09:57:09 | FromDiscord | <Elegantbeef> Imagine if you couldnt 😛 |
09:57:49 | FromDiscord | <Rika> Vim would be infinitely easier to exit |
09:58:16 | FromDiscord | <Elegantbeef> Hey all modern vim walks you out |
09:59:54 | arkanoid | mratsim, how did you became so skilled in low lever data handling? Academic background or else? |
10:00:01 | FromDiscord | <Rika> If it has a prompt telling me how to properly exit, why can’t it just… exit |
10:00:55 | FromDiscord | <Clonkk> What ?! Quitting is super easy `sudo pkill -9 vim`, boom you're done↵(@Rika) |
10:00:56 | FromDiscord | <Elegantbeef> I'm not UI-ologist↵(@Rika) |
10:01:36 | FromDiscord | <Rika> In reply to @Clonkk "What ?! Quitting is": Not in list of sudoers, incident reported, yada yada |
10:02:01 | FromDiscord | <mratsim> In reply to @arkanoid "<@570268431522201601>, how did you": I was bored. |
10:02:12 | FromDiscord | <Elegantbeef> I think you just kick the PC until the PSU cuts off |
10:02:15 | FromDiscord | <evoalg> at least vim gives you two different ways to exit |
10:02:36 | FromDiscord | <evoalg> oh three ways thx to beefy |
10:03:18 | FromDiscord | <mratsim> @arkanoid, @Clonkk @Vindaar ,↵↵Seems like Arrow is columnar only, i.e. it doesn't really support multidimensional Tensor: https://arrow.apache.org/docs/format/Other.html |
10:03:32 | FromDiscord | <Rika> I mean you could just wait for cosmic rays to kill vim with a memory error |
10:04:24 | FromDiscord | <mratsim> So you should be able to export/import with zero-copy but I can't use it for matrix or deep learning stuff. |
10:04:49 | FromDiscord | <evoalg> In reply to @Rika "I mean you could": as long as you have non-parity ram? |
10:04:55 | FromDiscord | <Clonkk> Well that sucks↵(@mratsim) |
10:05:15 | FromDiscord | <Rika> In reply to @evoalg "as long as you": Most don’t AFAIK, thanks Intel |
10:05:27 | FromDiscord | <Clonkk> Reminds me of the xkcd about the butterfly effect↵(@Rika) |
10:05:40 | FromDiscord | <Rika> They had a comic about it, huh |
10:05:44 | FromDiscord | <mratsim> In reply to @Clonkk "Well that sucks (<@570268431522201601>)": I do think it's a better data structure for dataframes though. |
10:06:02 | FromDiscord | <Clonkk> https://imgs.xkcd.com/comics/real_programmers.png↵(@Rika) |
10:06:12 | FromDiscord | <mratsim> representing nulls when you have an integer dataframes, or stuff that mixes integer and strings is such a pain. |
10:06:33 | FromDiscord | <mratsim> which is why I prefer to deal with deep learning 😉 |
10:06:45 | FromDiscord | <Rika> Emacs has the dumbest chord combo sometimes ngl |
10:06:50 | FromDiscord | <Rika> Can’t get used to it |
10:07:18 | FromDiscord | <Rika> I’ll be honest I have a harder time exiting Emacs than Vim |
10:07:19 | FromDiscord | <Clonkk> Yeah I'm mainly concerned about numerical Tensor↵(@mratsim) |
10:07:46 | FromDiscord | <Clonkk> Well that's how on purpose because emacs wants you to do everything inside it↵(@Rika) |
10:07:59 | arkanoid | Clonkk, without string manipulation, you're keeping a lot of science application outside nim domain |
10:08:01 | FromDiscord | <mratsim> In reply to @Clonkk "Yeah I'm mainly concerned": for numerical tensor the standard is DLPack which is the same repr as Numpy, Pytorch, tensorflow |
10:08:02 | FromDiscord | <Elegantbeef> Emacs is an OS not a editor |
10:08:24 | FromDiscord | <Clonkk> Yeah, it's just not my personal use case↵(<@709044657232936960_arkanoid=5b=49=52=43=5d>) |
10:08:41 | FromDiscord | <mratsim> In reply to @arkanoid "<@805869905320214558>, without string manipulation,": sure but you don't need 3D strings manipulation. |
10:08:41 | FromDiscord | <Clonkk> I don't disagree |
10:09:02 | FromDiscord | <mratsim> which is Arraymancer domain, say when dealing with colored images. |
10:09:18 | arkanoid | mratsim, see Apache Arrow Tensor https://arrow.apache.org/docs/format/Other.html |
10:09:30 | FromDiscord | <mratsim> The page is empty |
10:09:35 | FromDiscord | <mratsim> there is no format for tensors. |
10:09:56 | arkanoid | empty? I see it with multiple devices |
10:10:11 | FromDiscord | <Rika> https://media.discordapp.net/attachments/371759389889003532/932939917632348160/IMG_6167.png |
10:11:17 | FromDiscord | <mratsim> There is no format, <tensor body> doesn't tell me how to make that compatible with other implementations |
10:11:43 | FromDiscord | <Rika> Ah that’s what you mean |
10:12:02 | FromDiscord | <Elegantbeef> Yea it's a nothing burger without cheese |
10:12:13 | FromDiscord | <Elegantbeef> We give it padding, then wave our hands vaguely |
10:12:49 | FromDiscord | <Rika> I assume it’s as the prose says |
10:12:56 | FromDiscord | <Rika> Columnar spec with the change |
10:13:55 | FromDiscord | <Elegantbeef> > tensor (multidimensional array) stored as Binary values and having serialized metadata indicating the data type and shape of each value. This could be JSON like {'type'\: 'int8', 'shape'\: [4, 5]} for a 4x5 cell tensor |
10:14:33 | FromDiscord | <mratsim> yes but that's for IPC, there is no description of an in-memory representation. |
10:15:07 | arkanoid | it must be somewhere, arrow is made for interop |
10:15:26 | arkanoid | here's the C api (gobject based) for Tensor https://arrow.apache.org/docs/c_glib/arrow-glib/GArrowTensor.html |
10:15:34 | FromDiscord | <mratsim> And their columnar representation doesn't support Arraymancer use-cases. |
10:16:46 | FromDiscord | <mratsim> that's the same repr as Arraymancer. It doesn't seem to support null values or a mix of strings and ints and floats for example. |
10:17:13 | FromDiscord | <mratsim> so Arraymancer should be compatible with Arrow Tensor. |
10:17:41 | FromDiscord | <xx_ns> In reply to @Rika "If it has a": the same reason Ctrl-C doesn't close bash or zsh or any other interactive shell you're in |
10:17:48 | FromDiscord | <xx_ns> vim is also interactive, just like a shell |
10:18:11 | FromDiscord | <Rika> And plenty of other interactive programs do actually close with control+c |
10:18:40 | FromDiscord | <Rika> The shell idea is understandable, it’s the last program you want to close in a terminal |
10:18:54 | FromDiscord | <xx_ns> then those programs are incorrectly programmed 🙂 |
10:19:20 | FromDiscord | <xx_ns> if i press Ctrl-C, i want my half-completed command to be erased, not the entire shell closing |
10:19:29 | FromDiscord | <Rika> Then there are a majority more incorrect programs than “correct” ones I assume |
10:19:54 | FromDiscord | <Rika> Where did you get this convention anyway |
10:20:55 | FromDiscord | <xx_ns> i mean that's just how programs which provide a shell (like bash, zsh, vim, etc) work |
10:21:05 | FromDiscord | <xx_ns> if my shell exited after i press ctrl-c, i would be so mad |
10:21:45 | FromDiscord | <Elegantbeef> Let's take the pragmatic approach that ctrl + C should behave like `:q` in vim |
10:21:53 | FromDiscord | <mratsim> In reply to @arkanoid "here's the C api": Internally it's just that: https://github.com/apache/arrow/blob/36791327fabb0a137bb448dc10aeefa745640cf6/c_glib/arrow-glib/tensor.cpp↵↵It's pretty much this at the core: https://github.com/mratsim/Arraymancer/blob/master/src/arraymancer/laser/tensor/datatypes.nim |
10:21:55 | FromDiscord | <Elegantbeef> That way we can say it will warn you of work needing to be saved |
10:24:24 | arkanoid | mratsim: so you think Arraymancer 1D Tensor -> Apache Arrow Array, Arraymancer 2D+ Tensor -> Apache Arrow Tensor ? |
10:25:29 | FromDiscord | <mratsim> Datamancer -> Apache Arrow, Arraymancer -> Numpy/PyTorch/Tensorflow compatible repr like today. |
10:25:45 | FromDiscord | <mratsim> and glue for conversion |
10:26:41 | arkanoid | mratsim, but aren't Datamancer columns Arraymancer Tensors? Sorry if this sounds too naive, just asking |
10:27:08 | FromDiscord | <xx_ns> In reply to @Rika "Then there are a": also idk what you mean by this - i just tested, and pretty much all of the interactive programs i use regularly like vim, midnight commander, nmtui etc do not exit on ctrl-c |
10:27:09 | FromDiscord | <mratsim> They are, but they support multiple backends |
10:27:23 | FromDiscord | <xx_ns> even in the linux kernel's `make menuconfig`, ctrl-c does not exit by default |
10:27:39 | FromDiscord | <Rika> Maybe I’m misremembering then |
10:28:02 | FromDiscord | <Rika> I still don’t know why you would have each program exit a different way from every other program |
10:28:16 | FromDiscord | <mratsim> or I'm confusing with the other dataframe package |
10:28:36 | FromDiscord | <mratsim> anyway I think Arrow would be a better backend than Arraymancer cc @Vindaar |
10:28:52 | arkanoid | ok, but if I fill a Datamancer dataset in future Arrow mode, will I be able to run Arraymancer ops on single columns just like now? |
10:28:57 | FromDiscord | <mratsim> just for dealing with mixed data and nulls |
10:29:38 | FromDiscord | <mratsim> No because algorithms must take into account nulls and mixed values (string in a int column) so need to be adapted anyway |
10:30:02 | FromDiscord | <mratsim> if the datatype however is consistent yes |
10:30:03 | arkanoid | got it, I guess the differences are just like from pandas to numpy |
10:30:19 | arkanoid | btw, the pandas roadmap is filled with arrow references https://pandas.pydata.org/docs/development/roadmap.html#apache-arrow-interoperability |
10:30:26 | FromDiscord | <mratsim> you can use fromBuffer and point to the memory buffer of Datamancer/Arrow. |
10:30:33 | arkanoid | and pandas dtype string[arrow] already works |
10:31:45 | FromDiscord | <mratsim> basically, while at a high-level DataFrames and Tensors look similar. Tensors mostly deal with numerical types (except for text analysis but in that case characters are converted to integers) |
10:31:57 | FromDiscord | <xx_ns> In reply to @Rika "I still don’t know": again, that's just the convention for most interactive programs. I think why vim gets most of the heat for it is because `$EDITOR` is set to `vi` or `vim` on most distros by default |
10:32:10 | FromDiscord | <xx_ns> which, now that i think about, doesn't even make sense because even `nano` doesn't exit with ctrl-c! |
10:32:15 | FromDiscord | <mratsim> there are many special stuff that require special dance in dataframes. |
10:32:27 | FromDiscord | <Rika> In reply to @xx_ns "again, that's just the": I am fine with it not being C-c, I mean why are they all different in the end |
10:32:53 | FromDiscord | <Rika> In reply to @xx_ns "which, now that i": It shows how to at the bottom at all times |
10:32:55 | FromDiscord | <xx_ns> i mean `q` is the most used convention but there are exceptions |
10:33:16 | FromDiscord | <Rika> Vim does too but at the middle and it disappears if you type a bit and people are inattentive |
10:33:18 | arkanoid | mratsim, sure, I see each array/tensor as single dataframe column |
10:33:25 | FromDiscord | <Clonkk> From what I read, `GArrowTensor` could perfectly well be implemented as a `TensorConcept`↵(@mratsim) |
10:33:26 | FromDiscord | <xx_ns> In reply to @Rika "Vim does too but": vim shows it if you hit ctrl-c |
10:33:28 | FromDiscord | <xx_ns> since forever i think |
10:33:39 | FromDiscord | <xx_ns> i use neovim personally so i'm not too sure on that |
10:33:41 | FromDiscord | <Rika> Ah yeah that too but noobs don’t know of C-c either |
10:33:46 | FromDiscord | <xx_ns> or plain vi on embedded devices |
10:33:52 | FromDiscord | <Rika> Which is why they get flak |
10:34:03 | FromDiscord | <Rika> And probably a bit of bandwagoning effect too |
10:34:27 | FromDiscord | <xx_ns> well, "noobs" shouldn't be using `vi` 😁 and if they do get stuck, it's a quick crash course for them |
10:34:38 | arkanoid | mratsim, sure I've already experienced fromBuffer usage here: https://github.com/SciNim/scinim/issues/8#issuecomment-990573363, for numerical buffers zero-copy Arraymancer <-> Numpy <-> Arrow already seems to work |
10:35:02 | FromDiscord | <xx_ns> In reply to @xx_ns "well, "noobs" shouldn't be": anyways this topic is more suited for #offtopic probably |
10:35:02 | FromDiscord | <Clonkk> Dataframe column can change size while Tensor base element are always of the same size usually↵(<@709044657232936960_arkanoid=5b=49=52=43=5d>) |
10:35:11 | FromDiscord | <mratsim> sent a long message, see http://ix.io/3MEL |
10:35:20 | FromDiscord | <Rika> In reply to @xx_ns "anyways this topic is": I don’t think we should continue it anyway |
10:35:23 | FromDiscord | <xx_ns> same |
10:36:24 | FromDiscord | <Elegantbeef> Battling to the death would've been cooler |
10:36:38 | FromDiscord | <mratsim> https://tenor.com/view/game-of-thrones-i-demand-a-trial-by-combat-tyrion-lannister-gif-16328126 |
10:36:43 | arkanoid | Whats the advantage of Arrow vs DLPack? Why apache went from first to second? |
10:36:53 | FromDiscord | <mratsim> Apache went both. |
10:37:14 | FromDiscord | <mratsim> Arrow for DataFrames, DLPack for deep learning / multidimensional arrays / tensors |
10:38:07 | FromDiscord | <mratsim> DLPack doesn't need to deal with mixed datatypes (float+int+string), DLPack doesn't need to deal with empty values, and AFAIK it doesn't need to deal with strings. |
10:38:32 | FromDiscord | <mratsim> but it has to deal with multidimensional arrays. |
10:39:48 | FromDiscord | <mratsim> I think here it's important to specialize and provide glue between stuff. |
10:40:00 | arkanoid | I've worked a lot with pandas, and afaik each column (pd.Series) is a numpy array or ndarray with single dtype, can't fit multiple types in same column. But you can have null/mask |
10:40:55 | FromDiscord | <mratsim> you can fit multiple types, it just become "object" and use Python dynamic dispatch. Which you can't do in C/C++/Rust/Nim. |
10:41:10 | FromDiscord | <mratsim> they use that to good profit for null values |
10:41:21 | FromDiscord | <mratsim> that's why Apache Arrow validity map is interesting. |
10:41:40 | FromDiscord | <mratsim> it avoids dynamic dispatch in the common case of mixing integer + some missing. |
10:42:47 | FromDiscord | <mratsim> but asume you have integers and some missing, you can't use Arraymancer sum anymore unless you 0 the missing stuff. But in that case you can't use product, unless you "1" the missing stuff. |
10:43:16 | FromDiscord | <mratsim> so even algorithms need to be specialized to Apache Arrow/Dataframe specificities |
10:43:52 | arkanoid | in all my experience with multiple datasets and fixing datascience work written by non-programmers (been working in research lab for years) I've found almost zero Python Objects used as Dataframe cells, but yeah, lots of sparse Dataframes with nulls |
10:44:44 | FromDiscord | <mratsim> obviously you didn't get Excel data, where people are supposed to put a scale from 1 to 10 and suddenly you get "I took an arrow in the knee" |
10:44:45 | arkanoid | with pandas if you try to apply function to Series with nulls, you have first to replace them, or remove them, or impute them |
10:45:48 | FromDiscord | <mratsim> The worse being time field. Sometimes it's formatted as unix epoch, someties you get a random string. |
10:46:22 | FromDiscord | <Clonkk> unsanitized user input is hell to handle↵(@mratsim) |
10:46:41 | FromDiscord | <mratsim> so for data cleaning it's important to support mixed datatypes, which fortunately Arrow provide (except maybe on GPU) |
10:47:10 | arkanoid | well, the only reason python datascience world is live and kicking, is because numpy is forcing some static typing into python scripts, forcing order into chaos |
10:47:20 | FromDiscord | <Rika> In reply to @Clonkk "unsanitized user input is": Thank you for the idea: dataset from hell |
10:47:41 | FromDiscord | <Clonkk> Name of oyur punk rock band ?↵(@Rika) |
10:47:48 | FromDiscord | <Rika> Hey no lmfao |
10:48:53 | FromDiscord | <Clonkk> Well it was either that or your lastest thriller where a datascientist uses excel to solve murder |
10:49:32 | arkanoid | my experience is that pandas is used more for preprocessing "dirty" original data. Load dataset as is into pandas, turn strings into types, handle nulls, name columns -> start doing science |
10:49:44 | FromDiscord | <Rika> Imagine a world where every data set was denormal |
10:50:16 | FromDiscord | <Rika> No ID numbers, extra columns used for arrays, etc |
10:50:37 | FromDiscord | <mratsim> sent a long message, see http://ix.io/3MES |
10:52:35 | arkanoid | I think it would be really sweet. But what about strings? If Datamancer goes Arrow, it would be able to handle string data, but then a non-null string column would not be able to be converted zero copy to Arraymancer |
10:53:20 | arkanoid | btw, rust users uses polar, that based or arrow too > https://docs.rs/polars/latest/polars/ |
10:54:51 | arkanoid | Arraymancer currently uses nim strings as element, am I right? |
10:57:18 | FromDiscord | <evoalg> I'm glad I found this to capture ctrl-c https://stackoverflow.com/questions/54713213/how-do-i-handle-ctrl-c-interrupt-in-nim ... because when I look at https://nim-lang.org/docs/system.html#setControlCHook%2Cproc%29 it doesn't have an example and I would have been stuck. It seems a lot of things in system & manual for Nim don't have examples, and it forces babies like me to ask questions on here (which is something I've never had to do wit |
10:58:28 | FromDiscord | <mratsim> In reply to @arkanoid "Arraymancer currently uses nim": yes |
10:59:02 | FromDiscord | <mratsim> In reply to @arkanoid "I think it would": It can. Arraymancer is generic other the type. |
10:59:08 | FromDiscord | <mratsim> (edit) "other" => "over" |
10:59:39 | FromDiscord | <mratsim> If the Apache Arrow String has public proc, at least for `$` you can store them in an Arraymancer Tensor. |
10:59:53 | arkanoid | but with copy |
10:59:58 | FromDiscord | <mratsim> no |
11:00:50 | FromDiscord | <mratsim> you just hand the `buffer: ptr UncheckedArray[ArrowBuffer]` to Arraymancer |
11:02:01 | arkanoid | well, that would make Datamancer + Arrow a perfect marriage, and Arraymancer would still be capable to do all the low level superspeedy black magic it curretly does when dealing with multidimensional numerical tensors! |
11:02:02 | FromDiscord | <mratsim> basically, copy is necessary only for ownership. |
11:02:30 | FromDiscord | <mratsim> ownership is necessary with Nim strings/sequences or GC-ed types. |
11:02:52 | FromDiscord | <mratsim> or if the original owner disappears and delete its memory. |
11:04:10 | arkanoid | yeah I throw my zero-copy interop experiments to valgrind to double check this |
11:05:07 | FromDiscord | <mratsim> When I said, Arraymancer can be just the algorithm, I mean this: https://github.com/mratsim/constantine/blob/master/research/kzg_poly_commit/strided_views.nim↵↵This is a "strided view" for monodimensional arrays.↵It only neds memory to be valid while it processes it. |
11:05:17 | FromDiscord | <mratsim> (edit) "neds" => "needs" |
11:06:14 | FromDiscord | <mratsim> This is a generalization of openarrays ;). |
11:07:54 | arkanoid | yeah I've rolled my own views on seqs for a side project, too |
11:09:08 | FromDiscord | <mratsim> so do you want to tackle implementing the Arrow format? |
11:09:33 | arkanoid | btw, I'd really happy to be able to load and preprocess science data via nim Datamancer, and them once it is in shape pass it to Arraymancer ad tensor for real work |
11:11:35 | FromDiscord | <Clonkk> If you want to tackle a Nim arrow implementation we can add you to the scinim Team if you're interested↵(<@709044657232936960_arkanoid=5b=49=52=43=5d>) |
11:14:05 | arkanoid | do you mean building wrappers to Arrow C api, or reimplement pyarrow memory model structures in pure nim? |
11:15:48 | FromDiscord | <mratsim> In reply to @arkanoid "do you mean building": implementing in pure Nim.↵I've looked at the spec and it seems like the same amount of work to do any of those. |
11:16:14 | FromDiscord | <Clonkk> Ideally the second one as native implementation are often easier to work with but I can live with a good wrapper |
11:16:37 | FromDiscord | <mratsim> I'm sure people wants to ingest CSV at compile-time 😉 |
11:17:03 | FromDiscord | <mratsim> the issue with the first one is distributing the C files along in Nimble. |
11:17:16 | FromDiscord | <mratsim> does it work well in Flambeau btw? |
11:17:44 | FromDiscord | <mratsim> For example BLAS on Windows is just hell. |
11:18:34 | FromDiscord | <Clonkk> The distribution of the shared library ? So far I haven't had any issue, it's just super slow to download torch on installation. If you mean in general, it still need works to have a higher level API easy to use↵(@mratsim) |
11:19:30 | FromDiscord | <Clonkk> That said we could use a way to distribute artifacts with Nimble |
11:21:31 | arkanoid | a third option would be to use futhark to build C bindings automatically at compile time and build nim wrapper on it |
11:22:17 | arkanoid | sure I think a 100% nim implementation over stable Arrow C interface would allow compile time magic |
11:24:28 | FromDiscord | <Clonkk> Well you'd still to distribute the source code that Futhark needs as input ? At which point I don't really see the benefit of distributing C source file and wrapping them each compile vs just distributing the Nim generated file↵(<@709044657232936960_arkanoid=5b=49=52=43=5d>) |
11:27:33 | arkanoid | well, you surely can distribute the generated nim file, but the original idea behind futhark afaik was to make it transparent and follow C api changes. What futhark need is -dev package installed and the equivalent info given by "pkg-config --cflags --libs arrow-glib" |
11:30:57 | arkanoid | in my experience, it takes a couple of seconds to generate nim bindings for C interface, and it caches the result to avoid doing that a second time if hash of the sources doesn't change. PMunch is the man here |
11:32:00 | arkanoid | can nim do foreign function call at compile time? |
11:32:53 | PMunch | Thank you, I'm so glad you like Futhark :) And yes you are correct, the idea was to build against the same library as you would if you where writing a C program. If you have a different version installed then Nim won't just silently assume an old version and introduce potential bugs. |
11:33:02 | PMunch | arkanoid, no, that's why I wrote Opir |
11:33:26 | PMunch | Opir is compiled and links against Clang, then Futhark calls Opir to actually do the C file parsing |
11:34:50 | arkanoid | yeah offline Opir -> json, compile time json -> futhark -> nim |
11:35:34 | PMunch | Yup |
11:36:16 | PMunch | The idea was that Futhark should be able to react to things defined in your project, however this is done mostly with a lot of `when defined` statements. |
11:36:46 | PMunch | It does allow you to do things like a renaming/retyping callback proc and other such things though |
11:39:21 | arkanoid | yes, I've used that feature a couple of times when futhark didn't wrote the correct thing. It happens on const/enums |
11:41:04 | PMunch | Yeah there are a couple known weaknesses |
11:41:14 | PMunch | Hopefully I can iron out most of them after FOSDEM |
11:41:50 | arkanoid | apart from that, I never had a showstopper so far. The only one time I though it was one, it ended up being a nim regression |
11:42:15 | FromDiscord | <hugogranstrom> A C wrapper is better than no wrapper, but a pure Nim arrow package would be the best. Windows is awful when it comes to getting -dev packages and such. |
11:43:38 | arkanoid | I do agree, and I also consider it doable but I fear not having enough low level knowledge to build it with same quality as other scinim packages. I may try, I'm interested in learning, but I will require support and patience on your side |
11:47:48 | * | adigitoleo quit (Quit: adigitoleo) |
11:48:52 | FromDiscord | <vindaar> Seems like I've missed a lot these last couple of hours. Introducing a Arrow as a storage backend for Datamancer shouldn't be too much work, if the basic Arrow allocators are there. But it would certainly require us to use `fromBuffer` to hand the data to an Arraymancer tensor. Otherwise we end up missing things like `forEach` and add a few Arraymancer primitives (things like `concat` etc. that are used when resizing columns for example). |
11:49:17 | FromDiscord | <hugogranstrom> Quality-wise you can be rest assured you won't be too low, trust me 😉 If it works, it works. As simple as that 😄↵(<@709044657232936960_arkanoid=5b=49=52=43=5d>) |
11:50:43 | FromDiscord | <vindaar> So a separate `ArrowArray` thing that is used as a full replacement to the current Arraymancer.Tensor is a bit more work.Ideally of course we already want to have our concepts like `TensorConcept`, `VectorLikeConcept` etc. to have fully separate types (no `fromBuffer` needed), but still reuse most of the existing arraymancer, etc. code 🥰 |
11:52:47 | FromDiscord | <vindaar> (I'd vote for a pure Nim version too though) |
11:53:55 | FromDiscord | <Vindaar> In reply to @mratsim "We just need a": @mratsim already thinking about the important things, I see |
11:55:01 | arkanoid | point is: I basically know zero about the internals of Datamancer/Arraymancer, but if you say that I can start playing with Arrow data structures for nim without knowing them in advance, I'm in |
11:58:49 | FromDiscord | <vindaar> Internals about Datamancer / Arraymancer we can answer. Yeah, just start from the basic of implementing `type ArrowArray` (or whatever it may be called) according to the spec. That's needed anyway. Once that is implemented we worry about introducing it as a Datamancer backend! |
11:59:22 | FromDiscord | <vindaar> what's your github name again? |
12:01:13 | arkanoid | me: https://github.com/arkanoid87 |
12:02:23 | FromDiscord | <vindaar> ah, thanks |
12:03:06 | FromDiscord | <vindaar> you have an invitiation |
12:07:03 | arkanoid | thanks! |
12:22:12 | FromDiscord | <mratsim> In reply to @vindaar "Seems like I've missed": forEach needs special handling of missing values in a DataFrame package anyway no? |
12:23:33 | FromDiscord | <mratsim> also was forEach made public?It's backend agnostic and the main difficulty of forEach is dealing with multidimension, arrows is straightforward regarding that: https://github.com/mratsim/Arraymancer/blob/61188413abb269471ff5cbf4b9326ddd3c2a38c7/src/arraymancer/laser/strided_iteration/foreach.nim |
12:23:40 | FromDiscord | <mratsim> no "import tensor" |
12:25:17 | FromDiscord | <vindaar> well, `forEach` itself doesn't really care about missing values etc. If one has missing values in a `float` column, they are just NaN and it's up to whatever the user does whether that's a problem or not. If it's an `object` column, we use a regular `for` loop anyway |
12:25:59 | FromDiscord | <vindaar> ah, indeed. Didn't think about that as we have it as part of the arraymancer repo nowadays↵(@mratsim) |
12:26:20 | * | fvs joined #nim |
12:27:25 | FromDiscord | <mratsim> I think Datamancer can just use a for loop for everything. For parallelization, I'll think it over but I happen to have written so many threadpools in the past 2 years, maybe I'll be able to write them eyes closed xD |
12:28:19 | FromDiscord | <vindaar> sure, it could |
12:28:34 | FromDiscord | <vindaar> but as `forEach` works and is here, I don't see why not \:P |
12:29:36 | arkanoid | I fear that adding arrow for you is an easy task, so easy that my effor would end up just being a toy compared to what you would do in 10 mins |
12:29:39 | FromDiscord | <mratsim> well I'm not the one maintaining Datamancer 😉 :p |
12:30:19 | FromDiscord | <mratsim> I think it's a weekend task at least not the IPC stuff, just the data format. |
12:30:37 | FromDiscord | <mratsim> There might be some time spent figuring out the unions and strings and lists |
12:30:44 | FromDiscord | <vindaar> arkanoid "easy" is a very relative term. It may or may not be hard, but I agree with mratsim. It's at least a weekend project |
12:31:19 | FromDiscord | <vindaar> If it really was easy and little work, I would have done it already, btw (as arrow came up a few times before) |
12:31:42 | arkanoid | probably me again and again :P |
12:31:49 | FromDiscord | <mratsim> I don't think it's hard inthe sense that the specs spells out stuff quite clearly. |
12:32:56 | FromDiscord | <vindaar> not only you ;) |
12:33:00 | FromDiscord | <mratsim> I would do stuff with seq first, and then for the ptr UncheckedArray (otherwise no zero-copy possible and not compatible with C) we can figure that out later. |
12:38:48 | FromDiscord | <enthus1ast> in a macro, how can i get the object type implementation of a ref object |
12:39:33 | FromDiscord | <enthus1ast> on an object type i can just call\: ty.getTypeImpl() , but on a ref this returns\: ref Foo2\:ObjectType |
12:40:07 | FromDiscord | <enthus1ast> i want to get the referenced object type details |
12:49:09 | FromDiscord | <mratsim> In reply to @enthus1ast "i want to get": You sacrifice a goat to sigmatch.nim |
12:52:00 | FromDiscord | <mratsim> I think this works for ref objects: https://github.com/status-im/nim-stew/blob/8a405309c660d1ceca8d505e340850e5b18f83a8/stew/shims/macros.nim#L184 |
12:53:58 | FromDiscord | <enthus1ast> puh |
12:54:16 | FromDiscord | <enthus1ast> thank you @mratsim will have a look |
12:55:59 | FromDiscord | <mratsim> In reply to @enthus1ast "thank you <@570268431522201601> ": feel free to vent here once done: https://github.com/nim-lang/RFCs/issues/44 |
12:57:07 | FromDiscord | <mlokis> Would someone listen to my idea of algorithm? I usually find flaws when i talk about my idea. |
13:00:41 | FromDiscord | <mratsim> In reply to @mlokis "Would someone listen to": don't ask to ask, just ask 😉 |
13:01:20 | FromDiscord | <01> ye, just go ahead |
13:06:03 | FromDiscord | <mlokis> sent a long message, see https://paste.rs/H19 |
13:11:57 | * | rockcavera joined #nim |
13:11:57 | * | rockcavera quit (Changing host) |
13:11:57 | * | rockcavera joined #nim |
13:14:13 | FromDiscord | <mlokis> I came up with an idea that owner of the item should be the deepest dependency it has, so if you have modules `[a, b, c, d]` and connections `[a<-b, a<-c, b<-d, c<-d]` and you have tuple of `(i, j, k)` instantiated from `d` where `[i from a, j from b, k from b]`. Since there are no cyclic dependency allowed, either `b` nor `c` can possibly reuse the tuple. So we have two lowest dependencies and ownership of tuple will be taken by `d`. |
13:14:46 | FromDiscord | <mlokis> (edit) "b]`." => "c]`." |
13:18:27 | FromDiscord | <mlokis> @01 @mratsim is this explanation good? Does it make sense? |
13:19:29 | FromDiscord | <01> no clue |
13:20:41 | FromDiscord | <mlokis> In reply to @01 "no clue": i had one fundamental mistake here now its edited. Still nothing? |
13:22:15 | FromDiscord | <mratsim> so some kind of shortest path? |
13:22:25 | FromDiscord | <mlokis> no |
13:22:36 | FromDiscord | <mratsim> makes sense for me. That's what I do for task dependencies. |
13:22:48 | FromDiscord | <mlokis> finding shortest path is not a problem |
13:23:35 | FromDiscord | <mratsim> Ah no sorry, I randomize the owner: https://github.com/mratsim/weave/blob/master/weave/cross_thread_com/flow_events.nim#L54-L57 |
13:23:35 | FromDiscord | <mlokis> question is that lets say you make a change in `b`, you have to have guarantee that it has ownership of all types that depend on it |
13:24:10 | FromDiscord | <mratsim> so you have a direct acyclic graph? |
13:24:11 | FromDiscord | <mlokis> and that no two objects have ownership of same item |
13:24:21 | arkanoid | I have to handle a "not null-terminated string", not sure if handle it as a cstring and them remove the last byte, or go UncheckedArray[cchar] |
13:24:23 | FromDiscord | <mratsim> a tree if no cycle |
13:24:42 | FromDiscord | <mratsim> In reply to @arkanoid "I have to handle": seq[byte] |
13:24:48 | FromDiscord | <mratsim> or seq[char] |
13:25:04 | arkanoid | but I am writing a C interface |
13:25:10 | FromDiscord | <mratsim> if you use a cstring or a string you'll have a read past buffer at one point |
13:25:28 | FromDiscord | <mratsim> then openarray[char] |
13:25:38 | FromDiscord | <mratsim> that's converted to pointer+len |
13:25:47 | arkanoid | it's not an argument, but an object field |
13:26:41 | FromDiscord | <mratsim> cstring as fields will likely lead you to point to invalid memory. I would use ptr UncheckedArray[char] |
13:27:50 | FromDiscord | <mratsim> string processing in C requires to rewrite the whole stdlib anyway because relying on `\n` is just hyper inefficient |
13:27:59 | arkanoid | yeah, I went the same way. Thanks. Is there difference from char and cchar here? |
13:28:20 | FromDiscord | <mratsim> I don't know cchar |
13:28:32 | FromDiscord | <mratsim> there is already signed char and unsigned char so shrug |
13:28:48 | arkanoid | cchar* {.importc: "char", nodecl.} = char |
13:29:01 | arkanoid | in system.nim |
13:29:06 | FromDiscord | <mratsim> char is fine. |
13:29:55 | FromDiscord | <mlokis> nim being transpiler makes everything so weird |
13:30:16 | FromDiscord | <enthus1ast> 🤪 |
13:31:11 | PMunch | ... |
13:31:45 | FromDiscord | <mlokis> i think i isolated what the algorithms inputs and outputs should be so am gonna prot it in python and see. |
13:32:16 | arkanoid | if nim is a transpiler, also gcc and clang are a transpilers |
13:33:05 | FromDiscord | <enthus1ast> please no transpiler / compiler discussion again... |
13:33:38 | FromDiscord | <mlokis> In reply to @arkanoid "if nim is a": i don't think transpiling into ir that is designed for it and some programming language is quite same thing |
13:33:52 | FromDiscord | <mlokis> (edit) "In reply to @arkanoid "if nim is a": i don't think transpiling into ir that is designed for it and some programming language is quite ... same" added "the" |
13:33:53 | arkanoid | sorry, I never experienced such a discussion before here, I'll just say: badger badger mushroom |
13:34:58 | FromDiscord | <mlokis> why cant we play nim here |
13:35:39 | FromDiscord | <mlokis> am pretty sure you can make nim minigame with discord bot |
13:36:55 | FromDiscord | <mlokis> sent a code paste, see https://paste.rs/Dc3 |
13:37:45 | FromDiscord | <Rika> OH BOY |
13:37:53 | FromDiscord | <Rika> SOMEONE RESET THE TRANSPILER DAYS COUNTER |
13:38:17 | NimEventer | New thread by Planetis: How to modify and redistribute software written with MIT license?, see https://forum.nim-lang.org/t/8822 |
13:38:20 | PMunch | Haha, we should add that to NimBot :P |
13:39:13 | PMunch | By the way: https://peterme.net/is-nim-a-transpiler.html |
13:41:05 | FromDiscord | <enthus1ast> core wars with nim script would be awesome \:) |
13:41:16 | FromDiscord | <Rika> that would be hard |
13:41:28 | FromDiscord | <Rika> i mean i cant imagine how it would work |
13:42:26 | FromDiscord | <enthus1ast> yeah must be a little different, but the idea of let two (or more) nimscripts fight each other is tempting |
13:43:52 | PMunch | Core wars? |
13:44:02 | PMunch | Who's maintaining NimBot? |
13:44:31 | PMunch | And is the code public? |
13:45:38 | FromDiscord | <Rika> https://en.wikipedia.org/wiki/Core_War |
13:45:48 | FromDiscord | <Rika> cool little minigame pitting programs against each other |
13:46:26 | FromDiscord | <Rika> ok so what if |
13:46:32 | FromDiscord | <Rika> redcode nim backend |
13:46:50 | FromDiscord | <enthus1ast> or vm |
13:47:19 | FromDiscord | <enthus1ast> ah backend so that i can run on the MARS? |
13:47:19 | PMunch | Ah I see |
13:47:24 | FromDiscord | <Rika> or idk maybe just a program that can generate from nim-subset code |
14:08:18 | * | neurocyte091709 quit (Ping timeout: 268 seconds) |
14:08:35 | * | neurocyte0917090 joined #nim |
14:15:08 | arkanoid | mratsim, sorry for the naive question, but I'm unsure if I should write Arrow structs with manual memory management in mind (ptr instead of ref), or say in nimland and avoid unsafe |
14:19:31 | arkanoid | s/say/stay |
14:20:51 | arkanoid | I mean, the api says "It is intended for the base structure to be stack- or heap-allocated by the consumer. In this case, the producer API should take a pointer to the consumer-allocated structure." |
14:21:28 | * | arkurious joined #nim |
14:22:05 | arkanoid | the way I handle this depends on the requirements of the consumer, or generally the intended nim software design |
14:36:35 | FromDiscord | <mratsim> In reply to @arkanoid "<@570268431522201601>, sorry for the": If we want them to zero-copy into CuDF and other stuff from the Python ecosystem or C++ (say if gradient boosted trees package accept Arrow format) need to use the same in-memory representation as Apache Arrow, so manual memory management |
14:36:57 | FromDiscord | <mratsim> but that can be easily changed later. |
14:37:11 | arkanoid | ok, thanks, going into manual mode then |
14:38:25 | FromDiscord | <mratsim> I would go with Nim mode at first, layout the stuff. And then consider which libraries it's best to be compatible with (say CuDF, pandas/Dask?) |
14:38:43 | FromDiscord | <mratsim> and then look at those library what do they do with the arrow buffer we pass to them |
14:39:02 | FromDiscord | <mratsim> do they take ownership and in that case they need malloc-ed stuff from us. |
14:42:49 | szahid | is there something like "void" or none? |
14:44:01 | FromDiscord | <mratsim> sent a code paste, see https://play.nim-lang.org/#ix=3MG4 |
14:44:30 | FromDiscord | <mratsim> just an example, I don't have the full spec in my head. |
14:45:16 | FromDiscord | <mratsim> when we need to do manual memory management we can figure out what to replace ref ArrowDF if it's even needed |
14:45:27 | FromDiscord | <mratsim> and the buffer |
14:45:30 | FromDiscord | <Rika> In reply to @szahid "is there something like": in what context? yes there is void |
14:48:55 | arkanoid | mratsim: the docs clearly says that the consumer has to allocate memory on heap or stack and pass a pointer of it to the producer, that is responsible of filling it. On release, the consumer calls release on the struct he owns and then the producer chains all the release recursively until done |
14:55:59 | arkanoid | in manual mode would be "var foo: ArrowArray (or malloc if heap is desired); createArray(foo.unsafeAddr); foo.release", in nim mode would be completely different |
15:42:28 | FromDiscord | <01> anyone here knows Haxe? |
15:42:50 | FromDiscord | <01> is there a more widely adopted language than Haxe, that's meant to be compiled to a lot of targets? |
15:43:11 | FromDiscord | <Rika> why are you asking here |
15:43:13 | FromDiscord | <Rika> in a nim discord |
15:44:01 | FromDiscord | <enthus1ast> @01\: you could try nim |
15:44:03 | FromDiscord | <enthus1ast> \:) |
15:44:22 | FromDiscord | <01> sorry |
15:44:28 | FromDiscord | <01> i thought i did it on off-topic |
15:44:46 | FromDiscord | <Rika> well what's stopping you from using nim |
15:45:00 | FromDiscord | <01> small ecosystem |
15:45:05 | FromDiscord | <01> nothing else really |
15:46:42 | FromDiscord | <Rika> would you fancy rust or go |
15:46:49 | FromDiscord | <Rika> or zig i dunno |
15:46:54 | FromDiscord | <01> go is large, which is great |
15:47:06 | FromDiscord | <01> but a lot of interface{} |
15:47:06 | FromDiscord | <Rika> but? |
15:47:11 | FromDiscord | <01> from experience |
15:47:14 | FromDiscord | <Rika> yes |
15:47:18 | FromDiscord | <01> also poor support for fp |
15:47:19 | FromDiscord | <enthus1ast> c / c++ |
15:47:29 | FromDiscord | <enthus1ast> large mature eco system |
15:47:41 | FromDiscord | <01> ehh, yea, no |
15:47:49 | FromDiscord | <01> no standardized package manager/build tool |
15:47:50 | FromDiscord | <Rika> i would not wish such upon even my enemies |
15:47:51 | FromDiscord | <01> big time pass |
15:47:54 | * | toothlessgear quit (Remote host closed the connection) |
15:47:54 | * | Schnouki quit (Remote host closed the connection) |
15:48:08 | FromDiscord | <01> C/C++ is a mess imo |
15:48:24 | arkanoid | C++ is, C well, less |
15:48:27 | * | Schnouki joined #nim |
15:48:35 | FromDiscord | <Rika> not much less |
15:48:42 | * | toothlessgear joined #nim |
15:48:45 | arkanoid | yeah, but yet |
15:50:53 | FromDiscord | <enthus1ast> think c++ got a package manager recently |
15:50:59 | FromDiscord | <01> they have it sure |
15:51:04 | FromDiscord | <01> just not a standardized one |
15:51:06 | FromDiscord | <01> afaik |
15:51:15 | FromDiscord | <01> Conan, and whatever Microsoft made |
15:51:40 | FromDiscord | <01> vcpkg, whatever that is |
15:52:19 | FromDiscord | <enthus1ast> i still can recommend nim, the eco system is not too bad, if something is missing you could quickly write it yourself, or wrap a library. I personally can live quite good with nim |
15:54:36 | FromDiscord | <Rika> if you cant get over the issues of other languages' ecosystem then ignore it first and weigh the other issues |
15:54:52 | FromDiscord | <01> wdym |
15:56:18 | FromDiscord | <Rika> if the ecosystem of each language arent "desirable" one way or another, look at the other stuff |
15:57:05 | FromDiscord | <enthus1ast> What do you miss? |
15:57:10 | FromDiscord | <01> ecosystem is just about the most important part though |
15:57:42 | FromDiscord | <Rika> yes, but you cannot make a decision on it at all |
15:57:43 | FromDiscord | <Rika> so ignore it |
16:10:29 | FromDiscord | <mratsim> In reply to @arkanoid "<@570268431522201601>: the docs clearly": let's move to the #science channel, are you in it? |
16:11:26 | arkanoid | do you mean #nim-science? |
16:12:50 | arkanoid | well, maybe you're no talking about irc |
16:14:55 | FromDiscord | <Rika> yes |
16:15:00 | FromDiscord | <Rika> they mean in discord |
16:16:03 | arkanoid | never used discord in my life, worth trying? I'm too old for this new stuff |
16:17:02 | FromDiscord | <Rika> no |
16:17:17 | FromDiscord | <Rika> stay on irc if you have no issues with it |
16:18:39 | arkanoid | configuring discord, let's try |
16:18:51 | FromDiscord | <Rika> hes a goner.. |
16:19:10 | arkanoid | ? |
16:20:05 | FromDiscord | <Arkanoid> yo |
16:33:40 | FromDiscord | <Rika> hello |
16:59:41 | FromDiscord | <Patitotective> Is there another _npeg_ indentation example? https://github.com/zevv/npeg/blob/master/misc/indent.nim does not success (`ok=false`) and it doesn't actually capture the data into a table |
17:12:28 | FromDiscord | <gnu+linux user> In reply to @demotomohiro "You need to write": thonkuu |
17:41:44 | arkanoid | is it just me that once imported nimpy I got `.` template polluting the scope and I cant call normal nim procs anymore? |
17:55:52 | PMunch | Hmm, is there a way I can define something like `const ext* = <something>` so that `import ext/implementation` would import `implementation` from what `ext` was defined as? |
18:02:31 | PMunch | I guess I could create dummy files in `ext` which imported based on the same rule.. |
18:02:39 | PMunch | But that seems a bit tedious.. |
18:03:27 | nrds | <Prestige99> Hey PMunch, have you been running into issues with nimsuggest while using nimlsp recently? |
18:03:58 | PMunch | Nothing in particular comes to mind |
18:04:09 | PMunch | But my nimlsp might be compiled from an old version of Nim |
18:04:29 | nrds | <Prestige99> I've opened https://github.com/nim-lang/Nim/issues/19371 think it's been broken since 1.6.2 |
18:04:38 | nrds | <Prestige99> but I haven't heard of others having the same problem so far |
18:07:32 | FromDiscord | <demotomohiro> Create a macro that takes `ext: static[string]` and produce`import ext/implementation` statement? |
18:07:57 | PMunch | Prestige, full disclosure I barely use any of the NimLSP features :P |
18:08:54 | nrds | <Prestige99> D: |
18:09:44 | PMunch | @demotomohiro, well yes, I could always create a `importImplementation` macro which expanded to `import ext/implementation` |
18:10:37 | PMunch | But I want the user code to look like `import mylibrary; import ext/implementation` and mylibrary would then define `ext` in a way that the second import works |
18:10:39 | nrds | <Prestige99> It's just the diagnostics which are having issues, which are reported by nimsuggest. So likely just a nimsuggest bug |
18:10:49 | nrds | <Prestige99> or multiple nimsuggest bugs |
18:12:07 | PMunch | I could create all the files I want to define in a folder name `ext` which then imported mylibrary and then essentially turned into an include for whatever file I want, but that means I have to create all the files as dummies.. |
18:13:52 | PMunch | Prestige, it definitely looks like a nimsuggest bug |
18:17:37 | nrds | <Prestige99> well at least I'm not crazy |
18:17:55 | nrds | <Prestige99> hopefully someone looks into this before 2.0 lol |
18:18:49 | PMunch | Have you tried running a bisect on it? |
18:20:12 | nrds | <Prestige99> Not yet, just tried different nim versions and found it started after 1.4.8 |
18:22:15 | nrds | <Prestige99> I was going to bisect it but couldn't get nim to compile, will have to mess around with it a bit |
18:26:42 | PMunch | I wish there was an auto-bisect script shipped with Nim |
18:27:08 | PMunch | I had one that I used previously, but I think I've lost it.. |
18:31:38 | * | cyraxjoe quit (Ping timeout: 252 seconds) |
18:32:11 | * | cyraxjoe joined #nim |
18:50:27 | PMunch | Ugh.. |
18:50:55 | PMunch | Seriously, why can't I import the `os` module to use it in a macro when I'm compiling against AVR |
18:51:02 | PMunch | Such a stupid limitation.. |
19:07:19 | PMunch | Is there a way to have a package specific config file? |
19:08:01 | PMunch | I would like to have a config.nims file for my package that gets used when the package is used |
19:08:48 | * | jjido joined #nim |
19:15:48 | PMunch | There really should be a way to import a module only for use in macros |
19:15:53 | PMunch | Like `static: import os` |
19:28:08 | arkanoid | if I have an object attribute and a function taking that object with same name, and I want to keep them as is, how can I tell nim to call the function istead of getting the attribute? |
19:28:58 | PMunch | By using parenthesis |
19:29:54 | arkanoid | doesn't seem to work. I actually have an iterator named like an attribute of the object it iterates |
19:32:23 | PMunch | Hmm, curious |
19:32:27 | PMunch | I mean you can call it |
19:32:42 | PMunch | Like so: https://play.nim-lang.org/#ix=3MHL |
19:33:53 | * | krux02 joined #nim |
19:34:21 | PMunch | Oh great.. You can't put an include in a `when compiles` statement either.. |
19:34:40 | PMunch | So I can't hackily test if a file exists that way.. |
19:43:10 | PMunch | So I can't import os and check if a file exists, because of a stupid limitation. And I can't try to include the file and check if that compiles because of some stupid limitation. |
19:46:07 | PMunch | And I can't patch the os module from my library to use my own version instead. |
19:46:41 | PMunch | And copying the stuff in the os module doesn't work because it relies on the compiler to override the implementation, and that is done on a per-module basis |
19:49:54 | FromDiscord | <Patitotective> Are there some alternatives to _npeg_? |
19:50:13 | PMunch | @Patitotective, what are you looking for? |
19:50:24 | FromDiscord | <Patitotective> offside-rule |
19:50:31 | FromDiscord | <Patitotective> (edit) "offside-rule ... " added "(indentation matters)" |
19:50:53 | PMunch | You should be able to do that with npeg, no? |
19:51:41 | FromDiscord | <Patitotective> The _npeg_ example of indentation is poor so I don't know how to implement it |
19:52:15 | FromDiscord | <Patitotective> (edit) "it" => "a fully functional indentation parser" |
19:52:32 | PMunch | And instead of trying to figure it out you're looking for another parser which magically handles indentation? |
19:52:58 | FromDiscord | <Patitotective> In reply to @PMunch "And instead of trying": Both, meanwhile I'm reading this https://michaeldadams.org/papers/layout_parsing_2/LayoutParsing2-2014-haskell-authors-copy.pdf |
19:54:29 | PMunch | Hmm, if I remember my npeg right I think you basically need to have a matcher that matches zero or more " " at the beginning of your line, and then in the match you pass that and your state object to a procedure which counts the length of that match and figures out where it goes |
19:55:14 | NimEventer | New Nimble package! servclip - Manage your clipboard remotely, see https://gitlab.com/lurlo/servclip |
19:56:34 | FromDiscord | <Patitotective> sent a code paste, see https://play.nim-lang.org/#ix=3MHR |
19:57:39 | PMunch | Not quite sure I understand what you mean |
19:58:56 | FromDiscord | <Patitotective> In the example https://github.com/zevv/npeg/blob/master/misc/indent.nim there is only one level indentation, what happens with two or more? |
19:59:06 | FromDiscord | <Patitotective> (edit) "level indentation," => "indentation level," |
20:02:58 | * | noeontheend joined #nim |
20:03:30 | FromDiscord | <Patitotective> In reply to @PMunch "Not quite sure I": Should I share an example? |
20:12:27 | PMunch | It should work fine |
20:14:24 | PMunch | Hmm, it fails to match properly thought |
20:14:27 | PMunch | though* |
20:14:32 | * | rockcavera quit (Ping timeout: 240 seconds) |
20:15:43 | FromDiscord | <Patitotective> Also I would like to see the implementation to generate an actual table, but I guess it's kinda complex because it requires object variants |
20:15:48 | PMunch | I mean p.match(data).ok returns false |
20:18:11 | PMunch | It seems to fail on the `* !1` condition, that's strange |
20:19:02 | FromDiscord | <Patitotective> Maybe because it `&indentIn` and `&('\n' indentOut)`? |
20:19:14 | FromDiscord | <Patitotective> (edit) "Maybe because it `&indentIn` and `&('\n' indentOut)`? ... " added "(Doesn't consume the indentation)" |
20:19:19 | FromDiscord | <Patitotective> (edit) "indentOut)`?" => "indentOut)`" | "indentation)" => "indentation)?" |
20:19:24 | FromDiscord | <Patitotective> (edit) "(Doesn't" => "(doesn't" |
20:20:46 | PMunch | Hmm, that might be it |
20:22:41 | PMunch | Yeah, that seems to be it |
20:23:06 | FromDiscord | <Patitotective> And the solution is to remove the `&`, or? |
20:23:18 | PMunch | No, that makes it fail much earlier |
20:26:14 | PMunch | It almost handles two layers of indentation |
20:26:44 | FromDiscord | <Patitotective> In reply to @PMunch "It almost handles two": Why almost? |
20:27:03 | PMunch | You can't drop two levels at the same time |
20:27:31 | PMunch | This works: http://ix.io/3MI3 |
20:27:40 | PMunch | But remove the "t=10" statement and it fails |
20:30:06 | FromDiscord | <Patitotective> Maybe it should iterate backwards through `indentStack` until `$0` matches an element, and the index means the levels of indentation to get out from |
20:30:10 | FromDiscord | <Patitotective> (edit) "Maybe it should iterate backwards through `indentStack` until `$0` matches an element, and the index means the levels of indentation to get out from ... " added "(?)" |
20:30:42 | FromDiscord | <Patitotective> (edit) "Maybe it should iterate backwards through `indentStack` until `$0` matches an element, and the index ... means" added "it matched in" |
20:31:06 | PMunch | Yeah the fix is pretty trivial: http://ix.io/3MI4 |
20:31:33 | PMunch | Basically when getting out of indentation drop until the new indentation matches the current one |
20:31:54 | PMunch | In fact I'm pretty sure you wouldn't need the indentStack at all |
20:32:17 | FromDiscord | <Patitotective> In reply to @PMunch "In fact I'm pretty": Then? |
20:32:30 | PMunch | Jut use a number to track the current indentation |
20:32:39 | PMunch | I think that should work |
20:32:51 | FromDiscord | <Patitotective> Let me try |
20:34:39 | FromDiscord | <Patitotective> What is the opposite to the `inc` proc? |
20:36:02 | FromDiscord | <mratsim> In reply to @Patitotective "What is the opposite": dec |
20:36:18 | FromDiscord | <mratsim> you can also use inc(-1) I think |
20:36:22 | FromDiscord | <mratsim> or -= |
20:36:27 | FromDiscord | <mratsim> or += -1 |
20:37:18 | FromDiscord | <Patitotective> In reply to @PMunch "Jut use a number": What do you think? http://ix.io/3MI5 |
20:50:40 | PMunch | indOut could just also set the new indent level, maybe after verifying that the new level is lower than the old one |
20:53:09 | FromDiscord | <Patitotective> 👍 |
21:09:12 | madprops | patito detective |
21:13:32 | FromDiscord | <The Bron Jame (real)> How would I read input without making a newLine? |
21:19:09 | FromDiscord | <Elegantbeef> `stdin.readChar` though it's blocking |
21:26:29 | NimEventer | New thread by Reversem3: Nimble.directory down ? , see https://forum.nim-lang.org/t/8824 |
21:27:54 | * | PMunch quit (Quit: leaving) |
21:33:57 | FromDiscord | <inv> what is wrong here? https://media.discordapp.net/attachments/371759389889003532/933111994935038012/unknown.png |
21:34:13 | FromDiscord | <inv> autocomplete does not work |
21:34:39 | FromDiscord | <gerwy> In reply to @Patitotective "The _npeg_ example of": are you making a lang or something? |
21:34:51 | FromDiscord | <gerwy> because im making one too :3↵and its indentation based |
21:36:40 | FromDiscord | <Rosen> In reply to @inv "what is wrong here?": I also had it not work at all on windows |
21:36:50 | FromDiscord | <Rosen> Eventually gave up and just used vscode lmao but if you come up with a fix let me know |
21:38:10 | FromDiscord | <inv> @Rosen it is not windows. but WSL2 |
21:38:30 | FromDiscord | <Rosen> oh dayum |
21:38:53 | FromDiscord | <inv> and for windows - the same problem - nvim made coredump and decided than no way to avoid vscode. but decided to try one more time in WSL2 |
21:39:23 | FromDiscord | <inv> (edit) "and for windows - the same problem - nvim made coredump and decided than no way to avoid vscode. but decided to try one more time in WSL2 ... " added "and it does not work again. but vs+nim-plugin works fine again in WSL2 too" |
21:41:59 | FromDiscord | <IsaacPaul> Nim needs better error/error reporting for their tools. If autocomplete fails I'd like to see why. Perhaps I can open up the plugin and fix it. Currently using vscode without nimsuggest as that doesn't work for most of my projects. |
21:42:37 | FromDiscord | <Rosen> ok, so I copied the essence of that minimal config in my wsl2 env @inv and got autocompletion working correctly https://media.discordapp.net/attachments/371759389889003532/933114181354422272/unknown.png |
21:42:56 | FromDiscord | <Rosen> dumb questions that have to be asked, you did :PlugInstall, right? And you're trying to use it on a .nim file that's on disk? |
21:44:24 | FromDiscord | <inv> I did. hm, with your example - it stuck for first autocomplete, but started to work later |
21:45:09 | FromDiscord | <Rosen> strange |
21:47:21 | FromDiscord | <inv> it works and does not work the same line and char. brrr |
21:47:46 | FromDiscord | <inv> it should be group_by here, and I see it ... sometimes https://media.discordapp.net/attachments/371759389889003532/933115473057419324/unknown.png |
21:48:19 | FromDiscord | <inv> https://media.discordapp.net/attachments/371759389889003532/933115611532361818/unknown.png |
21:48:26 | FromDiscord | <Rosen> I'll try to get datamancer and repro that myself |
21:48:32 | FromDiscord | <inv> ctrl+space helps ... sometimes |
21:50:00 | FromDiscord | <Rosen> It's definitely that macro that's screwing it up |
21:50:16 | FromDiscord | <Rosen> Template, rather |
21:50:22 | FromDiscord | <inv> I do: `df.` then g - nothin, then r - nothing. then I delete last char , then ctrl+space - and I have it |
21:50:56 | FromDiscord | <inv> in vscode it works without any problem at all - that is why strange - I expected they use the same nimsuggesst |
21:51:05 | FromDiscord | <inv> (edit) "problem" => "problems" |
21:52:01 | FromDiscord | <inv> ah! looks like it is the same - but in vscode I type it faster 🙂 |
22:02:47 | FromDiscord | <Patitotective> In reply to @madprops "patito detective": just like conan |
22:03:34 | FromDiscord | <Patitotective> In reply to @Life Sucks "are you making a": I'm making a preferences system yaml-like↵It is based on a Python library https://patitotective.github.io/PREFS/ |
22:10:11 | * | rockcavera joined #nim |
22:10:11 | * | rockcavera quit (Changing host) |
22:10:11 | * | rockcavera joined #nim |
22:16:32 | FromDiscord | <gerwy> Ah nice |
22:21:25 | * | mahlon quit (Ping timeout: 256 seconds) |
22:22:03 | * | neocron quit (Quit: Leaving) |
22:24:11 | * | mahlon joined #nim |
23:00:33 | FromDiscord | <ajusa> Why does Nim get tripped up if I use the word include as an argument to a macro? Tried doing `foo(hx-include=5)` and it didn't compile. I know include is a keyword, but why is it an issue in a substring? |
23:01:03 | FromDiscord | <Elegantbeef> cause it's attempting to parse an include statmt |
23:01:04 | FromDiscord | <ajusa> Fwiw I used stropping to get the code working, but was curious as to why |
23:01:31 | FromDiscord | <ajusa> So include can't be used in any variables then? Or is it a macro thing? |
23:01:35 | FromDiscord | <Elegantbeef> Some keywords cause the parser to go "ok we're looking for a X statement" |
23:02:14 | arkanoid | I have defined an iterator that works when used like "for x in myIt(y)", but doesn't compile if "for i,x in myIt(y).pairs():" |
23:02:17 | FromDiscord | <Elegantbeef> sent a code paste, see https://play.nim-lang.org/#ix=3MIA |
23:02:18 | FromDiscord | <Elegantbeef> Since it's a `dotExpr` |
23:02:21 | arkanoid | what am I doing wrong? |
23:02:58 | FromDiscord | <Elegantbeef> `import std/enumerate` `for x, y in enumerate myIt(y)` |
23:03:11 | FromDiscord | <Elegantbeef> `pairs` is an iterator that is implemented for types |
23:04:11 | FromDiscord | <Patitotective> Hey @PMunch, I was wondering how to avoid that after you get out from two levels of indentation (to zero) the first level tries to indent out but it will fail since 0 is not less than 0↵http://ix.io/3MIz |
23:04:46 | FromDiscord | <Elegantbeef> you could also implement a `myItPairs(y)` which yields `(int, T)` |
23:06:16 | arkanoid | could you please expand a little bit what you mean with `pairs` is an iterator that is implemented for types? |
23:06:42 | FromDiscord | <Elegantbeef> https://nim-lang.org/docs/theindex.html#pairs |
23:07:41 | FromDiscord | <Elegantbeef> `pairs` is implicitly called when you do `for x, y in z` but it needs to be implemented for `z`, due to that you're using an `iterator` you'd either want to have a `pairs` variant or use enumerate |
23:08:40 | arkanoid | got it, thanks |
23:31:36 | FromDiscord | <Patitotective> Hey @enthus1ast, did you start with the char's table macro? |
23:32:10 | FromDiscord | <Patitotective> (edit) "Hey @enthus1ast, did you start with the char's table macro? ... " added "I don't understand it yet 😅" |
23:33:08 | FromDiscord | <Patitotective> (Context: I want to convert `"\\n"` into `"\n"`) |
23:33:44 | FromDiscord | <Patitotective> (edit) "(Context: I want to convert ... `"\\n"`" added "unescape the backslash to get the actual escape sequence:" |
23:33:49 | FromDiscord | <Patitotective> (edit) removed "convert" |
23:50:02 | * | noeontheend quit (Ping timeout: 240 seconds) |
23:53:08 | FromDiscord | <evoalg> @ElegantBeef is the reason that a lot of things in system don't have examples because Nim is a small community? I'm referring to when I wanted to use setControlCHook but I was completely lost as how to use it in Nim if it wasn't for the example of stackoverflow? In python it's really easy to look up examples of most things I might want, but as I say I know Nim isn't a big community. Unless I'm looking in the wrong place for setControlCHook |
23:54:30 | FromDiscord | <Elegantbeef> Kinda, someone probably didnt make an example, time for a PR from you |
23:55:22 | FromDiscord | <evoalg> so a PR isn't opening an issue? |
23:55:59 | FromDiscord | <Elegantbeef> A PR is a pull request, i'm suggesting you add a `runnableExample` to the `setControlCHook` |
23:57:40 | FromDiscord | <evoalg> oh ... I'm not experienced in using git ... I would somehow add it using git right? |
23:57:48 | FromDiscord | <Elegantbeef> Yep |
23:58:28 | FromDiscord | <Elegantbeef> You'd fork the compiler, go into the nim file you want to add the runnable example to, add it, make a commit pushing to your fork, then make a PR to the Nim repo |