<<30-04-2013>>

00:11:55*fowl joined #nimrod
00:15:25AraqI also have x11 crashes ...
00:36:28*Trix[a]r_za is now known as Trixar_za
00:39:25NimBotAraq/Nimrod 6ac07be Araq [+0 ±8 -0]: first steps to the expr/stmt unification
01:13:22*Trixar_za is now known as Trix[a]r_za
01:29:30reactormonkAraq, not anymore here
02:43:28*fowl quit (Ping timeout: 245 seconds)
03:07:35reactormonkAraq, nope, the codegen fails.
03:12:50reactormonkAraq, http://sprunge.us/SCjA
03:36:32*fowl joined #nimrod
03:52:39*OrionPK quit (Read error: Connection reset by peer)
07:48:35NimBotAraq/Nimrod f9522a1 Araq [+0 ±1 -0]: new js codegen: bugfixes
08:12:15*zahary_ joined #nimrod
08:22:10fowlmorning all. i've been reading a lot about different component system designs and implemented an rdbms-inspired design: https://gist.github.com/fowlmouth/5487130
08:26:23*Trix[a]r_za is now known as Trixar_za
08:27:00zahary_I've implemented an extremely efficient such system in C++. in the end it used space directly comparable to multiple inheritance and message dispatch speeds very close to virtual function calls
08:28:33Araq ## nimrod bug with `-`(a,b: set[t]) --> reported?
08:28:36zahary_most designs in the books and articles try to be simple to understand first and efficient second. in particular, they fail to exploit the important fact that most of the objects that will be created have share a common structure (every tank in the game is composed of the same set of components: Renderable,RigidBody,AIAgent, etc)
08:28:52fowlAraq: yes
08:29:11zahary_to exploit this you must have a type like TypeInfo that describes the layout of the actual instances and is shared between all of the instances
08:30:35Araqwhat's the advantage of a component system?
08:30:46fowlzahary: right, so all the possible base components are basically a columns in a table of entities, when a new entity is created its added to any systems that can support it
08:30:57zahary_the instance itself (entity) could be merely an array of the components that build it up (with their offsets specified in the TypeInfo) - in my system they were even directly constructed in a shared blob of memory
08:32:10fowlzahary: i plan on generating a tuple of the base components and an enum to identify them, in the same order, so entities[entity_id][CPosition] could be used to get to the position (maybe)
08:32:26fowlCPosition is also tied to `pos` for access in my other design so i'll keep that
08:33:11zahary_well, the entity object have to be polymorphic, it's inefficient if you reserve space for every possible entity in the tuple
08:33:26*q66 joined #nimrod
08:33:33zahary_a typical game ends up with about 250-500 component types
08:35:24fowlnah you just need a small set of inheritable components :)
08:35:29Araqzahary_: what's the advantage of a component system over MI or simply composition?
08:35:31zahary_Araq, I've described this before. it solves the problem that it's impossible to define a correct inheritance tree when there are multiple ways to discriminate between objects:
08:35:31zahary_renderable vs non-renderable, physical vs non-physical, user controlled, vs AI controlled
08:36:46Araqyeah but I can't see why composition can't work here
08:37:17Araqwell composition may create too big objects I guess
08:37:49zahary_it turns the problem around by basic everything on composition of traits. what is an enemy tank? it's renderable, it participates in path-finding, it installs sensors for vision in the spacial partitioning system, etc. each of this capabilities is represented by a separate component that can be added or removed to the type
08:38:06zahary_basing everything ...
08:38:13fowlzahary: please critique my implementation when you get a chance :) also if need be handling the rows polymorphically is only a step away imo
08:39:09zahary_fowl, I did critique it already - add a typoinfo type. make the entity a very cheap object using information from the typeinfo to support operation like "get component"
08:39:57zahary_add a vtable in the type info and write helper macros for defining procs working with entities that automatically dispatch to the right component
08:40:09fowlah im a bit leary of using typeinfo for runtime discernment of what components a type has
08:40:30zahary_my system had a notion of multicast messages - messages that get dispatched to every component that impelements them in the entity
08:41:06zahary_this proved to be very useful. for example all kind of component want to register shapes in the physics system for various purposes
08:41:42zahary_you have a single multicast message like "registerPhysicalShapes" and any component can plug into it
08:42:28zahary_most examples in the books use this style . Foo* f = entity.getComponent(Foo); f->DoStuff()
08:43:01zahary_I find this to be bad practice as it increases physical dependencies and coupling - we preferred strongly the entity based messages
08:43:38zahary_DoStuff(entity); // gets dispatched to the right component withing this entity, allows for polymorhic code just like virtual functions
08:44:01fowlwell i do not need to fetch components, being in the system guarantees the data is available
08:46:38zahary_wells, let's agree on the vocabulary here. what's an entity? this is an instance of something in the game world, right? one particular house, tank or player? so how do you get the 3D model of that particular house?
08:50:48fowlan entity is an index in the seq of data, you could do entity.rendr_data isa P3DModelInstance
08:51:04fowler entity_data.rendr_data isa P3DModelInstance
08:52:06zahary_alright, what I called entity so far is the TEntityData type in your design
08:52:33zahary_mare sure that it doesn't include space for every single component in the system, but rather only for those that compose this particular instance
08:53:36fowli dont think thats possible without using generic functions for everything
08:55:08zahary_it's possbile. getComponent becomes: entityData->components[entityData->typeInfo->componentOffset[ComponentID]]
08:56:02zahary_typeinfo is a fat object, but this is because there will be many instances having the same structure in the game - the most popular object type (a basic 3d model part of the scene) will have thousands of instances in the world)
08:56:38zahary_so all of these instances are cheap, but share the same typeinfo object
08:57:14fowlthousands of instances, which are references to one model, with a position component and maybe some color/texture info
08:58:42zahary_yes, and in you current design each one of them will be a tuple composed of 250 null pointers
08:58:54zahary_... and 3 non-null
08:58:58fowlyou have to cast to use the offset dynamically right
08:59:05fowler well i guess you'd have to cast anyways
08:59:15zahary_yes
08:59:48fowlok
09:00:27fowlcould be faster by defining entity types and saving the offset info outside of the entities
09:01:01fowlbut youd lose dynamic entity types ..hrm
09:02:01zahary_there is a way to define type composed of other objects - that are regular nimrod types - no need for an entity systems. The entity system allows you to treat all objects polymorphicly
09:02:40fowlyea i plan on doing all this with hefty macros
09:20:34AraqI still don't get it ...
09:20:51Araqso how are your entities laid out in memory, zahary_ ?
09:21:20zahary_mine system is evil. it allocates a blob of memory enough to store all components and uses placement new to construct them there
09:21:34zahary_it achieves dynamically the layout that the compiler will use in multiple inheritance
09:21:57Araqso you can keep them in array, right?
09:22:18zahary_in my system, the offset is a bytes offsets
09:22:52Araqyeah but *ultimately* are they kept in an array or in a list?
09:23:08zahary_what I proposed to fowl here is to do something simpler, and to skip the single-allocation blob and just use seq[PComponent] and offset within this sequence
09:23:42Araqwell I'm interested in your highly efficient way of doing it
09:23:59zahary_there is no array in my system. to get a component you do entityData->componentsStorageBytes[entityData->typeInfo->componentByteOffset[ComponentID]]
09:24:31zahary_during construction time of the typeinfo, the corrent aligned offsets are computed
09:24:34zahary_correct
09:25:31*Trixar_za is now known as Trix[a]r_za
09:26:13*xcombelle joined #nimrod
09:26:42Araqwell how do you iterate over all entities?
09:26:49zahary_each entity looks like this struct Entity { TypeInfo* typeinfo; void* storage; |
09:26:55zahary_}
09:27:58Araqnow we're getting somewhere ... keep it low level please ;-)
09:28:07zahary_iterating over all entities is something you may want to do, but this is not a concern of the entity system. in our project there were a World class that stored all the entities (also, they were stored in some spacial systems allowing you to determine visibility, raycast, etc)
09:29:40Araqshouldn't Entity directly contain the position btw? most entities are somewhere in the world
09:30:21Araqand if Entity is of a variable size, how do you store it in the "World" class?
09:30:37zahary_there are things like areas and wether objects that either have non-standard positions or no position at all
09:30:50zahary_so we had a separate component for the position
09:31:12zahary_WorldNode - almost everything has it
09:32:08fowlzahary: was this an in-house project
09:32:58zahary_it was an engine used for several games
09:36:27Araqso Entity is the tank, right? and that consists of several components?
09:36:49zahary_yes, a tank would be one example
09:37:21Araqso where does the void* storage point to?
09:37:57Araqthe | means it's a "struct" of variable size. right?
09:38:00zahary_storage is a memory blob laid out like this: [[WorldNode][3DModel][RigidBody]...]
09:38:32zahary_all the components that compose the instance in a single piece of allocated memory
09:38:43zahary_no | was a typo :)
09:38:53Araqah ... lol
09:38:55zahary_storage; }
09:39:10zahary_I meant that's the whole entity type - 2 pointers
09:39:39Araqand these are kept in an array in the world object?
09:40:00zahary_well, you can keep them whatever way you find useful in your project
09:40:14zahary_they are normal C++ types after all
09:40:39zahary_in most games you want to be able to find them spatially or to just update them on each frame somehow, so they end up registered somewhere
09:40:51Araqyeah but come on ... somebody needs to iterate over all tanks and updates their state ;-)
09:41:31zahary_in our World, they were kept in hashmap [ObjectID -> Entity*]
09:41:37zahary_that was used for iteration on each frame
09:43:03zahary_as you should see, this is something that depends on the game needs, not something that should be imposed on users by the entity system
09:44:10Araqyeah, got it
09:54:05Araq[[WorldNode][3DModel][RigidBody]...] # this means you keep all WorldNodes in an array, all 3dmodels in a separate array etc. right?
09:54:45zahary_no, I keep all components of a single Tank close to each other
09:55:11fowlzahary: would entities share 3Dmodel components or are they instances that reference the model
09:55:36zahary_my system also had another type of components called pinned components
09:56:05zahary_these were allocated separaqtely [[WorldNode][pointer to pinned Foo][RigidBody] ...]
09:56:53zahary_pinning was important issue, because it technically was possible to mutate instances at run-time by adding and removing component, but this required moving things in memory
09:57:04zahary_so pinned component guaranteed that they won't be moved around
09:57:41zahary_if you want to have all WorldNodes in a single array (for locality), what was possible is to use pinned components combined with free-lists for allocating them
09:58:30Araqthere was some pdf that suggested this locality is essential but I can't find it
09:58:37zahary_fowl, sharing was possible with pinned components that have refcounts
09:58:52zahary_yes, I've read it
09:59:06zahary_we exploited such locality in other places
09:59:21zahary_for example, on each frame you have to compute the final transform matrix for each object before rendering it
09:59:35zahary_many engines will store that in the WorldNode equivalent
09:59:57fowlwhat is locality ?
09:59:59zahary_but it's better if you store it in a per-frame allocated memory both for memory savings and improved locality
10:00:31Araqfowl: basically programming with memory caches in mind
10:00:33zahary_the memory savings come from the fact that you only need to compute it (and use memory for it) only for visible objects
10:01:00fowlah ok
10:19:36*Reiser quit (Ping timeout: 245 seconds)
10:23:20fowlhrm i am now getting this: `Error: internal error: wrong instantiated type!
10:23:21fowlNo stack traceback available`
10:23:52Araqso the typeinfo describes what components an entity consists of and at which offset to find it, right?
10:25:54fowlye\
10:27:04*Reisen joined #nimrod
10:27:38zahary_Araq, yes. it also holds a big vtable used when messages are dispatched to components
10:31:37Araqwhy do you need this? you can easily do: fn(entity[positionOffset(entity.typeInfo)]) ?
10:32:44Araqyou can get away with static dispatch with your scheme
10:33:16zahary_because of the coolness of the multicast messages that I described and the possibility for polymorphism (one message implemented by different components in different instances)
10:34:12fowli changed up TSystem a bit and now i get this weird internal error: wrong instantiated type! with no trace .. https://gist.github.com/fowlmouth/5487130#file-components2-nim-L47
10:34:44zahary_in theory, a very efficient system will be one that allows you to define various type of messages - multicast, statically dispatched, dynamically dispatched and they all will use the same syntax in the user code so it will be very easy to switch the way given message is dispatched
10:34:58zahary_entity.foo should be the syntax everybody uses
10:35:08Araqfowl: well I'm afraid that's a tough one ... :-/
10:36:06fowlError: unhandled exception: f.sons[0].kind == tyGenericBody [EAssertionFailed]
10:36:52fowlheres the trace https://gist.github.com/fowlmouth/5487911
10:37:08fowlyou should switch koch to a nakefile (:
10:44:25Araqfowl: edit compiler/sigmatch.nim line 564
10:44:38Araqer like 556
10:44:43Araq*line
10:45:08Araqinstead of the assert do:
10:45:22Araqif f.sons[0].kind != tyGenericBody:
10:45:29Araq debug f
10:45:37Araq debug a
10:45:42Araq assert false
10:46:41Araqand then you only need to find out how a tyGenericInvokation can ever be build without a tyGenericBody at position 0 ;-)
10:49:11fowli was specifying the generic params explicitly
10:49:20fowllol that assert false made the compiler not recompile
10:53:33Araqhow can that be?
10:54:11Araqit bootstraps in debug mode so obviously f.sons[0].kind == tyGenericBody in the compiler itself everywhere
10:54:56fowloh lol i didnt see the if line -_-
10:55:08zahary_this often happened to me in the beginning - making some change that make the compiler unusable
10:55:44zahary_https://gist.github.com/zah/5487992 it's faster to build a debug version and use that for testing usually. these are some helpers I use
10:56:02Araqkoch boot ensures you always have a working compiler
10:56:17zahary_yes, but it could be brutally slow if you do something stupid
10:57:04fowler ok so the problem is this TActiveSystem*[userData] = object of PSystem i thought that was Okay
11:00:01Araqit is okay, the compiler has a bug obviously
11:00:53fowlo
11:00:55Araqon the other hand your userData stuff cries for a closure
11:03:46fowlid still need a separate type for systems that return something
11:15:45Araqzahary_: did you do any measures how your component engine performs against a hash based scheme? in particular I wonder how lua tables + luajit would perform for this use case
11:17:19Araqan entity can simply be a lua table that contains all its components and access is uniform: entity[Component]
11:17:31zahary_my engine used only fixed offsets for all operations and didn't waste space as a hashtable would
11:17:57Araqtrue but you have an indirection via the typeInfo to get the offset
11:20:10zahary_I guess seq[PComponent] could be more easily comparable to a hashtable, but I cared for the single memory allocation too so I didn't even consider such design
11:22:54Araqhow's the debugging experience when all your data is in a void* btw? ;-)
11:23:22zahary_visual studio allows you to write a primitive form of debugging visualizers
11:24:04zahary_http://www.idigitalhouse.com/Blog/?p=83
11:24:24zahary_we used these to make it display a tree of the components
11:24:35zahary_but it's essential to be able to do this indeed
11:40:28fowli am trying to get around this bug by nesting the the types, its not working >_<
11:58:11Araqfowl: tried to make TSystem generic as well?
12:13:17fowli think the problem was that i was still referring to PSystem[T] in a proc
12:14:30Araqaha
12:14:41Araqwell that sounds much easier to fix in the compiler then :P
12:15:04fowlyes when i get this to compile ill try it again the original way to make sure
12:22:41Araqugh!
12:22:45Araqthis works ...
12:22:51Araqproc retInt(x: int): int =
12:22:53Araq result = case x
12:22:55Araq of 23: 3
12:22:56Araq of (var yy = 1; for i in 0..1: yy *= 8; yy): 2
12:22:58Araq else: 1
12:23:13Araq--> the compiler happily computes yy at compile time for the case branch ...
12:24:45Araqoh well it reports a non-const expr correctly ...
12:25:01Araqquite impressive
12:28:09fowlAraq: ok no error now
12:28:24Araqfowl: report it anyway
12:28:38fowli recall seeign an issue like this already
12:37:39fowlno thats a diff issue
12:41:11NimBotAraq/Nimrod 4854859 Araq [+0 ±3 -0]: proper scoping for 'if'
12:41:28Araqwell the new (;;) stuff works now afaict
12:41:31Araqbbl
12:44:56zahary_merge the new branch already :)
12:48:58fowlagreed
14:15:13reactormonkAraq, http://sprunge.us/SCjA
14:24:31*Reisen quit (Ping timeout: 264 seconds)
14:29:30*Reisen joined #nimrod
14:54:12Araqreactormonk: fixed it already
15:11:40*zahary_ left #nimrod (#nimrod)
15:31:00*xcombelle_ joined #nimrod
15:33:25*xcombelle quit (Ping timeout: 248 seconds)
15:55:20reactormonkgood
16:13:23*Trix[a]r_za is now known as Trixar_za
16:23:17Araqugh, I broke the tester?
16:24:38Araqoh I see ... hrm
16:24:40Araqping zahary
16:54:42*Amrykid quit (Ping timeout: 264 seconds)
16:56:32*Boscop quit (Quit: Boscop)
17:02:04*Amrykid joined #nimrod
18:29:00*Boscop joined #nimrod
18:44:01*Trixar_za is now known as Trix[a]r_za
19:02:53*xcombelle_ quit (Remote host closed the connection)
19:33:02reactormonkAraq, what's the state of covariance?
19:33:16reactormonkmaybe relevant http://twitter.github.io/effectivescala/#Types%20and%20Generics-Variance
19:35:40Araqthe state is as it was before: covariance/contravariance doesn't work for value based datatypes and it doesn't really work with containers without lots of complexity and so I don't give a shit
19:36:30Araqand btw C# got it right before scala I think
19:38:23reactormonkthat's possible
19:57:19*Trix[a]r_za quit (Ping timeout: 256 seconds)
19:57:47*Trix[a]r_za joined #nimrod
20:16:32fowlim going to start writing hideous code and posting it to the pascal and basic forums
20:17:06fowlIF MYVAR.ODD:
20:17:23fowl FOR x,y IN foo.items:
20:17:23Araqand accomplish what, fowl ? ;-)
20:17:51fowlget someone to run something hopefully
20:18:12fowlinfect them with my fowliness
20:19:46*gradha joined #nimrod
20:24:01gradhathe threads module says the different memory model improves efficiency and the channels module says the implementation is slow
20:24:05gradhaso which one is it?
20:24:52Araqwell both are correct
20:25:02Araqthread local heaps are nice for performance
20:25:14Araqdata exchange sucks though
20:25:52gradhaso nimrod is better for low communication parallelization?
20:25:57Araqif you know what you're doing you can pass 'ptr's between threads
20:26:18Araqand get back the efficiency
20:26:47Araqthe missing piece is fast shared memory ... I'm working on it
20:27:07Araqwell the design is finished in my head ... ;-)
20:27:53gradhaI was planning on throwing idetools external execution into a thread pool and see the fireworks
20:29:53Araqwell in fact, there also is a shared heap already ... it's not GCed though
20:30:03Araqthe profiler uses it
20:31:52Araqgradha: use osproc parallel process execution instead for idetools
20:32:33Araqthe compiler can't be threaded easily
20:34:09gradhaI'll do this manually to report progress reports, but will take a look at execProcesses() code
20:34:54gradhathis is the kind of API which would gain a lot through a user supplied proc
20:36:50Araqhmm true
20:37:08Araqbut then you have to decide which effects the callback may have
20:37:57fowlwhat is allowed inside expr[T]
20:39:29Araqfowl: don't use expr[T] please for now; it will be renamed to static[T]
20:40:44Araqhmm effect propagation for callbacks is actually quite simple to implement ...
20:41:00Araqyay ... another todo list item ...
20:42:45gradhacan you specify "you can pass any proc to this function except those generating X,Y and Z effects"?
20:43:06Araqthat's a planned feature, currently it can't be done
20:44:42Araqgradha: what's the use case?
20:44:53Araqwhat effects do you want to rule out?
20:44:59gradhadon't have any
20:47:03gradhain the old days you could implement functions called in interrupt contexts and you wanted them to be fast, maybe disallow IO effects and similar?
20:48:00Araqyeah but then you want to disallow FTime and FLua (invokes Lua scripting somewhere) too
20:48:14Araqso you end up with tags:[]
20:49:06Araqif you want to restrict effects you often really want to allow what you in fact considered valid
20:49:31Araqand so you whitelist the allowed effects instead of blacklist the undesired effects
20:50:33Araqit's hard to come up with a use case for blacklisting effects especially since the set of effects is open
21:07:41*Amrykid quit (Changing host)
21:07:41*Amrykid joined #nimrod
21:11:30Araqif (let m = input =~ re"abc"; m.isMatch):
21:11:37Araq echo m[0]
21:11:38Araqelse:
21:11:50Araq echo m[1] # error: 'm' not declared
21:12:10Araqfine with everybody?
21:12:27Araqin other words the scoping rules for 'if' are slightly subtle :P
21:12:41Araqto support the above common use case
21:12:52fowlyea
21:12:57fowli lik eit
21:12:59gradhalooks weird when you "unroll" the if
21:13:24Araqit's crap if you have 'not' condition ;-)
21:13:34fowlif(let (good, res) = someFunc(); good): res.stuff
21:14:02Araqthe real reason for supporting that is of course not 'if' but 'elif'
21:14:36fowlmore parens = more accidental smileys
21:14:40Araqbecause for 'elif' you can't simply move it out of the condition
21:15:14Araqno smileys for me here
21:15:53fowlAraq: sometimes i'd love to be able to use _ and digits to join idents
21:16:17fowlat least inside `s
21:16:26Araqhuh? you can do that already?
21:16:45gradhaisn't in essence the if only checking the value of the last expression? sounds like you could rewrite that if construct as:
21:16:49gradhaprevious_statements
21:16:56gradhaif last_statement:
21:16:58gradha blah
21:17:00gradhaelse:
21:17:02gradha meh
21:17:08fowlAraq: no i mean proc `foo _ 42` = ...
21:17:09Araqgradha: read what I wrote :P
21:17:12fowlnot `foo_42`
21:17:36Araqtry the same for 'elif' to see why it's nice to have
21:19:30gradhadon't understand, isn't the if of the elif just appended into the first else block for chaining?
21:19:51Araqelif (let c = f(); c): ...
21:19:53Araqbecomes:
21:19:56Araqelse:
21:20:01Araq let c = f()
21:20:04Araq if c: ...
21:20:37Araqdo that a few times and watch your indentation explode
21:20:53Araq'elif' is much nicer for chaining
21:21:17gradhaerrr... wait, what I was meaning is: "isn't nimrod already doing that internally?"
21:22:02Araqyeah in fact you could do the same with a helper template
21:22:11Araqnow we have an explicit syntax for it
21:24:02fowlcan i use a template anywhere i use a type or should i just use a type alias
21:24:13Araquse a type alias
21:27:04Araqfowl: what is the use case for `foo _ 42`
21:27:41gradhairc code formatting sucks so I ellaborated my question on https://gist.github.com/gradha/5492095
21:28:08fowlAraq: i was writing a template to generate accessors with importcizzle
21:28:54fowldefgetter(name, ty): stmt = proc `get_ name`*: ty
21:29:27Araqgradha: you're correct: the expansion changes scoping
21:29:37fowlimportc needed the name to be get_*
21:30:06AraqI still don't get it fowl
21:30:28Araqimportc: "get_$1" ?
21:30:55fowloh ._.
21:32:37*gradha quit (Quit: bbl, have youtube videos to watch)
21:54:25NimBotAraq/Nimrod 56e86ed Araq [+0 ±4 -0]: tester should work again; documented (;)
21:56:24*fowl quit (Read error: Connection reset by peer)
22:14:11*Roin quit (Ping timeout: 245 seconds)
22:26:54*OrionPK joined #nimrod
22:28:24NimBotAraq/Nimrod e3e6edf Araq [+0 ±1 -0]: re.nim compiles again