GDR Forum Index
Podcast Podcast
Dev Dev Logs
Search Search
RSS RSS
Register Register
Log in Log in
Reply to topic GDR Forum Index -> Game Developer's Refuge -> Development Log - GEL and Foundation (Engine Tools) Page 1, 2, 3, 4  Next
View previous topic :: View next topic  
Author Message
PoV
Moderator

Joined: 21 Aug 2005
Posts: 10792
Location: Canadia
PostPosted: Sun May 02, 2010 9:33 pm    Post subject: Development Log - GEL and Foundation (Engine Tools) Reply with quote

Okay, enough of this not blogging and not doing.

The Foundation is my new cool inspirational name for my upcoming engine development project. It's the continuation of my Playground engines project from 2007. Playground is where I changed my development philosophy that any element written for an engine should be written in such a way that it can be gutted easily, or inserted in to completely different projects with minimal dependencies. Obviously that's not practical for everything, but a mistake I made in the development of Hammer (the engine for the shelved PuffBOMB HD prototype) was I couldn't really take and re-use any of the collision detection code, the physics engine, the polygon triangulation code, etc. It's one big self referential mess.

A timeline of names I've used for my engines follows.

- Brown Box - 2003-2004 - 3D Physics - Unreleased Digital Illusions R&D Physics Project
- Phiz - 2003-2004 - 2D Physics - Zooble, Phiz/Canadianese Simulator, Water Sim
- Destructure - 2004-2005 - Integer based 2D platformer engine with verlet physics - Atomic Betty (GBA)
- Freedom - 2005-2006 - 2D Physics - The Spider (Prototype), Ballistic Force (Prototype)
- Hammer - 2006-2007 - 2D Physics with 3D Backgrounds - PuffBOMB HD (Shelved)
- Playground - 2007-2010 - 2D Engine and Game Framework - Smiles, Smiles HD, numerous prototypes (Magtraction, Dungeon Legion's swarming test)
- Playground 3D - 2009 - 3D Engine - Dungeon Legion (Unfinished/Unreleased), "Mike learning how to use Shaders"
- The Foundation - 2010+ - TBD

Prior to 2003, my engines were designed with a specific game in mind. In the case of the above, the engines were often started/designed before a projects was officially assigned to them. Only 2 of the above have shipped titles associated with them (Destructure and Playground).

Goals for The Foundation
  • Cross Platform (inherited from Playground)
  • Mobile to HD resolution graphics, in both wide and tall aspect ratios (inherited from Playground)
  • Code tools for UI/Screen creation (inherited from Playground, to be enhanced)
  • Native streaming music support (inherited from Playground (1 track at a time), to be enhanced)
  • Native and ADPCM compressed sound effects (inherited from Playground, though currently uncompressed on most platforms)
  • Custom 2D texture container format (inherited from Playground, to be enhanced)
  • Able to mix and match 2D and 3D content
  • Fixed function and shader based materials pipelines (inherited from Playground 3D, though complete rewrite expected)
  • 3D decals (inherited from Playground 3D, to be enhanced)
  • 3D internal co-ordinate system, designed with a "2D Optimized" mode for systems lacking SIMD support
  • Modular elements that can be used with minimal dependencies (inherited from Playground)
  • Improve the ability to quickly prototype new ideas

  • Multiple collision systems, usable all at the same time. Tile based, polygon based, physics based, soft body, etc.
  • "World", which contains "Rooms", which contain objects, collision systems, and various zones and nodes
  • Integrated Map Editor
The big notable change to what I'm doing is I'd like to start using a 3D internal co-ordinate, even if the game is 2D, but write it in such a way that I could enable a "2D Optimized" mode. Even in Smiles, I implemented several "hack" ideas that would have been cleaner had I just included a Z. I'll need to be sure my volumetric tests (rectangles, spheres) do >= and < ranges, and so on.

Also, as mentioned in another thread, I'd like to stay playing around with something Metroidvania/platformer like. And despite all the 3D stuff talked about above, I'm not really aiming (yet) to make a 3D game, but I'd like the option. I have a bunch of 3D code written and reintegrated, just unused. Mainly, I'm just mentioning my intent to continue enhancing the 3D side.

In a way you could say this stuff is already started and in progress, since this is directly continuing with where Playground (currently in Smiles) leaves off. Though I probably wont be starting on any Foundation specifics until I get a few more Smiles ports and SKU's out the door, I sooo want to. Heh. So I'm hoping I start doing some planning and sharing here, between all the porting.
_________________
Mike Kasprzak
'eh whatever. I used to make AAA and Indie games | Ludum Dare | Blog | Tweetar


Edited by PoV on Tue Jun 01, 2010 8:16 pm; edited 7 times
View user's profile Send private message
Sirocco
Moderator

Joined: 19 Aug 2005
Posts: 9413
Location: Not Finland
PostPosted: Sun May 02, 2010 9:47 pm    Post subject: Reply with quote

Quote:

Even in Smiles, I implemented several "hack" ideas that would have been cleaner had I just included a Z.


Yup. That was a lesson I learned the hard way several projects ago. It's a lot easier to zero out a number you don't need than it is to hack one in every time.

That aside, this all sounds very ambitious. Me? I'm lazy!
View user's profile Send private message Visit poster's website
PoV
Moderator

Joined: 21 Aug 2005
Posts: 10792
Location: Canadia
PostPosted: Sun May 02, 2010 9:59 pm    Post subject: Reply with quote

Ambitious yes, but that's normal for me.

But if anything, not much has really changed from what I was doing anyways. I just figure the Playground name is 3 years old now, and has several shipped products to its name (all derivatives of the same SKU, but still). Switching to a 3D co-ordinate is a surprisingly big change, and it can be inspiring to name something. So a new name for the new effort, snce I'm really not sure what project I'm doing next.

This engine stuff is technically all encompassing of my prior agenda, just I think I want to start thinking about platformers again.

The other thing I want to put some effort in to is making it so I can quickly prototype a game idea. Right now, I don't have that. I have a bunch of useful stuff from fonts to layout creation, but last time I tried, it took me like 20-30 minutes take what I have and start something new. On the other hand, I can write up a simple basic Allegro program from memory in about a minute, and start experimenting. That has to change.
_________________
Mike Kasprzak
'eh whatever. I used to make AAA and Indie games | Ludum Dare | Blog | Tweetar
View user's profile Send private message
Sirocco
Moderator

Joined: 19 Aug 2005
Posts: 9413
Location: Not Finland
PostPosted: Mon May 03, 2010 6:15 pm    Post subject: Reply with quote

Quote:

I have a bunch of useful stuff from fonts to layout creation, but last time I tried, it took me like 20-30 minutes take what I have and start something new.


I sorta have this problem with my framework as well. I don't think I can improve on that without serious effort, and given how infrequently I experiment I don't see it being a problem. You... however... yes, you have crazy ideas and need to be able to check them out on a whim.
View user's profile Send private message Visit poster's website
sonrisu
Moderator

Joined: 31 Aug 2005
Posts: 4979
Location: Silicon Valley!
PostPosted: Mon May 03, 2010 8:21 pm    Post subject: Reply with quote

Every time I tried this I realized that when I come up with a game idea, after the fact, it had some really specific feature that would never be fulfilled by a general purpose all-around engine.

The Physics stuff seems cool. Actually everything sounds just about right except for the last three parts. They seem so game-specific! Then again, I know from experience that I rewrite this kind of thing all the time. Better to get it done once and for all, right?

Anyway. Seems cool. One thing you seem to be missing is resource management though. :)
_________________
loomsoft :]
View user's profile Send private message Visit poster's website
PoV
Moderator

Joined: 21 Aug 2005
Posts: 10792
Location: Canadia
PostPosted: Mon May 03, 2010 10:08 pm    Post subject: Reply with quote

Whoops, I guess I never saved that edit. The last 3 were supposed to be under a separate heading, a test project to start developing with Foundation.

And I agree you don't want to make an engine that does everything, but perhaps my definition of engine is also misunderstood.

I'll come back to this... I have to go take this moment and go cheer a lot.
_________________
Mike Kasprzak
'eh whatever. I used to make AAA and Indie games | Ludum Dare | Blog | Tweetar
View user's profile Send private message
Gil
Developer

Joined: 14 Nov 2005
Posts: 2341
Location: Belgium
PostPosted: Tue May 04, 2010 7:23 am    Post subject: Reply with quote

Instead of an engine, it's supposed to be more of a rapid development, multi platform deployment thingy, right?
_________________
PoV: I had to wear pants today. Fo shame!
View user's profile Send private message Visit poster's website
Sirocco
Moderator

Joined: 19 Aug 2005
Posts: 9413
Location: Not Finland
PostPosted: Tue May 04, 2010 9:04 am    Post subject: Reply with quote

Quote:

but perhaps my definition of engine is also misunderstood.


I don't suppose I've ever followed industry norms as far as nomenclature goes. File it under laziness if you will. I throw around two terms, and will now shamelessly offer my personal definitions of said items, for no good reason:


- Engine

An extensive and specific set of code that allows for the construction of a complete game, as long as it falls within the genre-specific capabilities of the code. This is great if you are going to rattle off sequels or similarly designed work. Jeff Vogel of Spiderweb is surely familiar with this approach. Think of this as a big LEGO set that allows for the construction of any type of train you can imagine, but only a train.


- Framework

A potentially (hopefully!) lightweight collection of related pieces of code that allows for more rapid development, by implementing commonly used components in a flexible manner. The pieces of code should be genre as well as platform agnostic as much as is reasonable, and designed in a non-restrictive manner. Think of this as a small tub of LEGO bricks with your favorite shapes and part combinations, thoughtfully arranged to interact with one another, along with any other LEGO sets you feel like adding in.



My theory at this point is that PoV is aiming more for the latter category, whatever he chooses to call it.
View user's profile Send private message Visit poster's website
PoV
Moderator

Joined: 21 Aug 2005
Posts: 10792
Location: Canadia
PostPosted: Tue May 04, 2010 10:26 am    Post subject: Reply with quote

Yes, there's actually 2 projects going on here, but yes, "The Foundation" is miss-labeled as an engine. The engine is actually the 3 point project below the framework, but one of the oddball goals of Playground (and thusly Foundation since it's a derivative) was to build an engine creation framework. Drop in engine components if you will.

For example, there's nothing that says a tile renderer can't be treated the same way as a 3D model: textures and geometry data, it's just how it's interpreted that's different. Same thing goes for tile-based collision data.

So the engine project is a simple implementation of a set of systems. Best case, if even a level editor was modular, it could be used and reused, as could elements to make new editors (actions and undos).

So it's not only about creating an engine, but distilling aspects of an engine down to components, so they can be mixed and matched elsewhere.

I'm typing this on an iPod, so I want to stop now. ;)
_________________
Mike Kasprzak
'eh whatever. I used to make AAA and Indie games | Ludum Dare | Blog | Tweetar
View user's profile Send private message
PoV
Moderator

Joined: 21 Aug 2005
Posts: 10792
Location: Canadia
PostPosted: Tue Jun 01, 2010 9:36 pm    Post subject: Reply with quote

Good morning!!

Thinking out loud. No need to actually comment, but if you have something to add, feel free. I normally write this stuff privately (my work routine often involves much ranty brainfart note-taking), but I guess I don't really need to. ;)

- - - -

I will begin reorganizing my engine library thingy soon. Here's what my source directory looks like right now.



If you're looking at the left side, you'll note there's more platform specific code in the Target folder, as well as the iPad folder (since I was too lazy to move it).

The goal of "Smiles: Phase 2" was to add support for a crap-ton more platforms to my framework. To get a real idea of how to structurally support a crap-ton of platforms in one codebase, by actually doing it. And of course, earning some money and prizes along the way thanks to the porting effort. :)

Platforms Supported:
- PC (MinGW and MSVC)
- Mac (i386 only)
- Linux (Debian (Ubuntu) or Fedora based)
- Moblin (Intel's Netbook Linux)
- Maemo (Nokia's Mobile Linux on the N900)
- MeeGo (Intel and Nokia's new joint Linux project)
- webOS (Palm's custom Linux on the Pre and Pixi)
- iPhone and iPod touch
- iPad
- Bada (Samsung's OS)
- Windows Mobile 6.5 (legacy)
- Android (compiling only)
- Google's Native Client (compiling only)
- ??? Console (compiling only)

Graphics API's Supported:
- OpenGL 1.5 and OpenGL ES 1.1 (Fixed Function)
- OpenGL 2.1 and OpenGL ES 2 (Shaders)
- Direct3D Mobile (legacy)
- Direct3D 9 (stubs, but some placeholders from the D3DM research)
- ??? Console (stubs)

Sound API's Supported:
- SDL_mixer
- IrrKlang (unsure about it's future)
- OpenAL (junky iPhone/iPad use, to be obsoleted)
- Custom Mixer (currently used on Windows Mobile and Bada, may replace some API's)
- ??? Console (stubs)

But yeah, lots of code. Every platform listed supports at least 1 modern 3D graphics API (no more software rendering) and has floating point numbers, though GCC does support emulating floating point (older Android handsets like the G1 don't have an FPU, and neither does the DS). I used to have a Fixed point math library back when I did GBA stuff, and I still currently use a generic type "Real" instead of float (making switching it possible), but I think it's a safe bet every platform I care about from now on will do floating point (or fake it via GCC).


Once I finish some obligations, I'll be doing some reorganizing of the above.

That's a lot of folders in my source directory, which makes it hard to find what I'm looking for (usually the game folder). So I need to structure this better.

The new layout I'm considering is as follows.

Code:
/DRM/
/DRM/AtomStore/
/DRM/Palm/
/DRM/Steam/

/Foundation/
/Foundation/MainLoop/
/Foundation/UI/
/Foundation/.../

/Game/

/GEL/
/GEL/Platform.h
/GEL/AdvancedGeometry/
/GEL/Chunk/
/GEL/Core/
/GEL/Core/C/
/GEL/Core/???/
/GEL/Core/Unix/
/GEL/Core/WinCE/
/GEL/Core/Windows/
/GEL/External/
/GEL/External/boost/
/GEL/External/lzma/
/GEL/Geometry/
/GEL/Graphics/
/GEL/Graphics/???/
/GEL/Graphics/Direct3D9/
/GEL/Graphics/Direct3DM/
/GEL/Graphics/OpenGL1/
/GEL/Graphics/OpenGL2/
/GEL/Graphics/OpenGLEE/
/GEL/Grid/
/GEL/Math/
/GEL/Rebel/
/GEL/Set/
/GEL/String/
/GEL/Timer/
/GEL/Timer/???/
/GEL/Timer/Bada/
/GEL/Timer/Unix/
/GEL/Timer/Windows/
/GEL/WhitespaceTokenizer/

/Main/
/Main/SDL/
/Main/???/
/Main/Bada/
/Main/NACL/
/Main/Android/
/Main/iPhoneOS/
/Main/WinCE/


DRM is exactly that. Sure, everyone hates it, but selling in some stores requires it.

GEL stands for Game Engine Library. It's something I started with Smiles, but kind-of let get structurally messy, as I tried to figure out the right way to organize it. Originally this was just the graphics library, but my input library currently (temporarily) lives alongside graphics.

The "/libc" folder in the image is my "File IO and serialization" library. Things like compressing or decompressing a file I consider standard features, alongside bulk reading an arbitrary file in to memory (DataBlock* BinaryFile = new_DataBlock("blah.bin");). This is something I'm now calling "GEL Core". It needs cleaning up right now. It hackily supports reading files and directories, but I'd like to clean this up to make it easier to add new input sources. Specifically, I'd like introduce the concepts of "Content" and "Storage", since every platform seems to have these concepts, and reading/writing to both aren't the same (memory cards, local storage, remote storage, etc). I'd also like to add some extensions for structurally supporting Content creation, mainly to be used on PC, but that's a whole separate topic.

The "/Shared" folder in the image above is my "everything else" library. Originally I was going to isolate different platforms ways of supporting getting files and stuff (see /libPackage), but I never did. Instead, I simply added hack files inside the "/libc" folder. Everything else got put in "/Shared". My vector and matrix math code, geometric operations like solving a sphere versus a polygon, parsing a text file as tokens/opcodes, among other things. I will be collapsing all of this in to GEL.

Finally, the "/Graphics" folder in the image above is my work-in-progress refactoring of my graphics code. In getting the "???" and OpenGL ES 2.0 port up and running, it became obvious that I have a lot of common code and structure between platforms. How I render a line on each API is different, but drawing an X or an Arrow is the same on them all (draw 2 or 3 lines). Also, how I load a PNG file would be the same on most platforms, so there's no reason to have that twice. Uploading to VRAM, that would change however. I plan to move the individual implementations out of separate folders, in to sub folders of Graphics. And depending on the target platform, the correct one of those is used. Finally, I'm planning to make Graphics in to "GEL Graphics", since it already that anyways (all my API's have already been renamed to gelCommand(...);).

Platform.h is something I want to introduce to clean up GEL. Right now, supporting each new platform is a matter of adding a bunch of #DEFINES to either the makefile or the project file for each target. That's fine, but there's some weird ambiguity to deal with. For example, Windows CE (Mobile 6.5) can read and write files the same way as Unix/Linux, but it needs to read directories differently. I'd simply like to structurally clean this up more. This is mainly part of the Content and Storage restructuring I mentioned above, but it comes up in Timers and Graphics too. Slight variations in initialization code on webOS versus Maemo versus SDL proper.

GEL is my fundamental cross platform library. All code that may be specialized for a specific platform lives inside GEL. GEL isn't just low level code, but high level geometric operations too. Eventually I may support stuff like geometrically testing a bunch of objects against eachother in general purpose shader code, or at the very least, optimizing my basic tests for various SIMD implementations (SSE, NEON, SPE, ALTI-VEC). Either way, all common math stays in GEL.

Foundation is what I should have talked about above, but kinda got lost in describing. Essentially, the hub for higher level "code tools" for creating things. Right now it's empty, but I'll be moving some of my code from Smiles in to it (my UI stuff for example). I've noticed also that I seem to write mainloops roughly the same way, but slightly different each time. So I may add some helper-tools for mainloop structuring. Finally, eventually, I'd like to think I might build some general purpose content editors, or stuff for making tools (UNDO's, scripting, etc). Or in summary, things that will never need to be low level rewritten for a platform. Macro operations and such.

Main contains actual mainloops. Just cleaning up some anomalies like /SDL, /WindowsMobile, /Android, etc. I'd rather not make them a requirement of GEL. GEL I want to be "things you may want to use, but don't have to". Mainloops are kind-of required, and force stuff to happen.

Then if all is done right, Game should contain several calls to Foundation elements. Right now with Smiles, there's lots of explicit calls to draw code. I don't expect to remove them all (don't care that much), but if I've done something right, it should be standalone enough to be part of Foundation.


Anyways, that's what's going on in my brain at the moment: structure. And heh, now that it's written down, I should be able to sleep. :)
_________________
Mike Kasprzak
'eh whatever. I used to make AAA and Indie games | Ludum Dare | Blog | Tweetar
View user's profile Send private message
Gil
Developer

Joined: 14 Nov 2005
Posts: 2341
Location: Belgium
PostPosted: Wed Jun 02, 2010 5:13 am    Post subject: Reply with quote

That post is like sex to me.
_________________
PoV: I had to wear pants today. Fo shame!
View user's profile Send private message Visit poster's website
Sirocco
Moderator

Joined: 19 Aug 2005
Posts: 9413
Location: Not Finland
PostPosted: Wed Jun 02, 2010 5:24 am    Post subject: Reply with quote

Quote:

That post is like sex to me.


Heh... your OCD is showing ;)
View user's profile Send private message Visit poster's website
sonrisu
Moderator

Joined: 31 Aug 2005
Posts: 4979
Location: Silicon Valley!
PostPosted: Wed Jun 02, 2010 10:14 am    Post subject: Reply with quote

Nice. It's always really refreshing to fix up your project layout. The only downside is that it's a major pain in the ass -- especially if you have complex dependencies. While my framework is still doing pretty good in that department, NF is currently out of control when it comes to actual structure on the disk. It's scary.

In any case, I feel your pain with figuring out how to lay things out for multiple platforms -- and I'm only targeting Win/Lin/OSX. I can imagine how much more of a PITA it is when you have to use a completely different graphics pipeline.

#define GOOD_TIMES
_________________
loomsoft :]
View user's profile Send private message Visit poster's website
n29
Developer

Joined: 13 Sep 2005
Posts: 879

PostPosted: Wed Jun 02, 2010 3:58 pm    Post subject: Reply with quote

My only comment is that 'deep' directory structures for source code sucks mega donkey. IDE support is the only thing that makes it bearable in Java (where package structure == directory hierarchy). That's why I can't use Vim as a Java editor; I've never found a Vim plugin that makes working with Java source as intuitive as the typical Java IDE.
View user's profile Send private message
sonrisu
Moderator

Joined: 31 Aug 2005
Posts: 4979
Location: Silicon Valley!
PostPosted: Wed Jun 02, 2010 4:03 pm    Post subject: Reply with quote

There's a directory browser in Vim, isn't there? I thought so. It's not the best (not like an IDE), but it's better than absolutely nothing. I yearn for the day I can use Vi as an editor in Xcode... built into the editor window. That would be amazing. ;)
_________________
loomsoft :]
View user's profile Send private message Visit poster's website
PoV
Moderator

Joined: 21 Aug 2005
Posts: 10792
Location: Canadia
PostPosted: Wed Jun 02, 2010 6:05 pm    Post subject: Reply with quote

n29 wrote:
My only comment is that 'deep' directory structures for source code sucks mega donkey.

I agree, but I have over 700 source files to deal with here, 80 in the "libc" (becoming GEL/Core) folder alone. And that's even with the occasional "multiple classes per file" going on. ;)

Directory wise it shouldn't be more than 3 levels deep though. I don't use an IDE (other than to build certain targets), so IDE limits are never an issue. :)
_________________
Mike Kasprzak
'eh whatever. I used to make AAA and Indie games | Ludum Dare | Blog | Tweetar
View user's profile Send private message
PoV
Moderator

Joined: 21 Aug 2005
Posts: 10792
Location: Canadia
PostPosted: Wed Jun 02, 2010 9:02 pm    Post subject: Reply with quote

June 3nd - 1:01 AM
Created a new SVN branch and got started on this crazy re-factoring.

Moved all the legacy code (and stuff I just don't want to deal with right now) in to a "_Legacy" folder. That's mainly Windows Mobile/WinCE stuff, but that also included the Direct3D 9 port and a custom library for Palm that's about to be obsoleted.

Did something similar with Graphics. Moved the GL2 renderer and ??? graphics code in to a "_Temp" folder, so I can deal with just the fixed function stuff right now.

And after much fiddling, the layout now reads very much as described above. :)

Code:
/src/DRM/
/src/External/
/src/Foundation/
/src/Game/
/src/GEL/
/src/Main/


A minor change: both GEL and the root tree have an External folder. The GEL version are external libraries required by GEL, such as LZMA and boost (select headers only). The root External are things not required by GEL, though some items there are used by GEL. IrrKlang, BZIP, Khronos EGL, STLPort, and ZLib right now (all optional things in the mind of GEL).

Creating a target in the old setup involves:
1. #defining a bunch of macros like "USES_UNIX" and "USES_OPENGLES"
2. Adding source directories to the project/makefile of the elements you want. i.e. "src/OpenGL1", "src/libc", etc.
3. Adding include paths for all these directories as well. "-I src/OpenGL1", "-I src/libc".

The 3rd one is so that I can say <Graphics/Graphics.h>, and that'll be the correct one depending on my renderer used.

The new setup will work the same way, but including "GEL" take the place of like 5 of those. Structurally speaking also, the old "src/Graphics" folder contained a "/Graphics" folder of it's own, so the path would resolve as <Graphics/Graphics.h> seen above. Now the files normally found under "src/Graphics/Graphics" will be directly under "GEL/Graphics/". And you add the entire GEL to a project.

That does mean that every file for every platform will need to be included and compiled. That means every piece of platform specific code needs to be wrapped with a define that includes it only if needed. I.e. #ifdef USES_WINDOWS_TIMER. This does makes source files a little messy, but the other option, having to explicitly include/exclude files I found to be even more annoying. What's more important: all the files on the same subject are in the same place, so there's no hunting for files. That's where large source trees with unwieldy hierarchic layouts fail, when you can't find anything.

Heh, so as is, my new source tree will not compile. I've broken the layout of things soooo much. :)

I started out by setting up 1 piece of GEL just right, as a proof of concept. Timers.

(Note: This layout evolved as I wrote this, which is exactly why I write these things in such silly detail, so I can question/rationalize my decisions).

Code:
/GEL/Platform.h
/GEL/Timer/
/GEL/Timer/Timer.h  **
/GEL/Timer/Timer_Functions.h
/GEL/Timer/Bada/BadaTimer.cpp
/GEL/Timer/Bada/BadaTimer_Type.h
/GEL/Timer/???/???Timer.cpp
/GEL/Timer/???/???Timer_Type.h
/GEL/Timer/Unix/UnixTimer.cpp
/GEL/Timer/Unix/UnixTimer_Type.h
/GEL/Timer/Windows/WindowsTimer.cpp
/GEL/Timer/Windows/WindowsTimer_Type.h


I avoided this setup initially because I wanted things like the timer code to stay standalone, but micromanaging the elements individually and by themselves is a real pain in the arse. The file you want here is "Timer.h", and depending on the platform configuration, the correct timer code will be included and built.

I still want it to be possible to use the timer code isolated from GEL, so I added a new define "NO_PLATFORM_H". If that's defined before including the file (or on the command line), the default behavior of including "Platform.h" at the top will be ignored. That then means you'll explicitly have to say which timer you want (USES_UNIX_TIMER, etc...).

The individual timers are not isolatable anymore. UnixTimer.cpp for example now relies on "../Timer_Functions.h" to compile. I figure there's no point in maintaining separate interfaces that are all identical. That was the point after all, to have a common interface to several ways of getting and calculating periods of time. So instead of multiple timer libraries, one for each platform, you have a timer library for multiple platforms.


Each _Type.H file simply defines the type used to describe a unit of time.
Code:
// -------------------------------------------------------------------------- //
#ifndef __UNIXTIME_TYPE_H__
#define __UNIXTIME_TYPE_H__
// -------------------------------------------------------------------------- //
#include <sys/time.h>
// -------------------------------------------------------------------------- //
typedef struct timeval TIMEVALUE;
// -------------------------------------------------------------------------- //
#endif // __UNIXTIME_TYPE_H__ //
// -------------------------------------------------------------------------- //

Some platforms use a time type. Others use a mere integer or 64bit integer.

Each CPP file is a body of code like the following.
Code:
// -------------------------------------------------------------------------- //
#ifdef USES_UNIX_TIMER
// -------------------------------------------------------------------------- //
#include "../Timer.h"
// -------------------------------------------------------------------------- //
// ... code goes here ... //
// -------------------------------------------------------------------------- //
#endif // USES_UNIX_TIMER //
// -------------------------------------------------------------------------- //

It includes Timer.h, since that will correctly resolve to the right Type header include, as well as the function prototypes. I originally was going to include Timer_Functions.h explicitly in the CPP files, but I realized it wouldn't necessarily resolve correctly. For example, if I was to define "USES_WINDOWS", that's something picked up by Platform.h that says "If I'm using windows, then I obviously need USES_WINDOWS_TIMER". So to be sure my CPP file picks up this logic, it's including the general header (Timer.h).

Also I opted to call the files WindowsTimer_Type.h, since that's all they actually contain (the type). If it was self sufficient (including function prototypes) it would be called WindowsTimer.h. In fact, it used to be called that. But now it's part of Timer general, so it's lost its individuality.

That also brings up my underscore rule. All of my names used are in StudlyCaps, but I bring out the underscores for specializations. In some cases that's files that are just too huge, I put class init code in it's own file (Game_Init.cpp). In this case, it's because it is incomplete (not WindowsTimer.h). I use them for complex function namings too, like Test_Sphere_Vs_SphereCluster... assuming a SphereCluster was something meaningful.


Anyways, that's my rant of rationalization. I seem happy with it now, and I think these are the general rules I'll be following for the other subsystems and platforms.

Time for sleep!
_________________
Mike Kasprzak
'eh whatever. I used to make AAA and Indie games | Ludum Dare | Blog | Tweetar


Edited by PoV on Wed Jun 02, 2010 9:20 pm; edited 2 times
View user's profile Send private message
sonrisu
Moderator

Joined: 31 Aug 2005
Posts: 4979
Location: Silicon Valley!
PostPosted: Wed Jun 02, 2010 9:14 pm    Post subject: Reply with quote

Quote:
That does mean that every file for every platform will need to be included and compiled. That means every piece of platform specific code needs to be wrapped with a define that includes it only if needed. I.e. #ifdef USES_WINDOWS_TIMER. This does makes source files a little messy, but the other option, having to explicitly include/exclude files I found to be even more annoying.

#define all the way. You are right that it can make code kind of messy, but, I also find it worth it.
_________________
loomsoft :]
View user's profile Send private message Visit poster's website
PoV
Moderator

Joined: 21 Aug 2005
Posts: 10792
Location: Canadia
PostPosted: Wed Jun 02, 2010 9:21 pm    Post subject: Reply with quote

Yeah, it's kinda nice to find all the Audio code under *GASP* the AUDIO folder. ;)
_________________
Mike Kasprzak
'eh whatever. I used to make AAA and Indie games | Ludum Dare | Blog | Tweetar
View user's profile Send private message
PoV
Moderator

Joined: 21 Aug 2005
Posts: 10792
Location: Canadia
PostPosted: Thu Jun 03, 2010 2:38 am    Post subject: Reply with quote

June 3rd - 6:38 AM
Woke up (too) early and made a rather insignificant change (effort wise). Timer.h now expects "USES_GEL" to be defined, otherwise it will not include "../Platform.h". I think I like this better than the NO_PLATFORM_H define, as it explicitly requests it like other features are requested (USES_WINDOWS, etc).

If I do it right, the elements of GEL will not require the USES_GEL define, but that define will simply tie the whole library together better.

I guess it's time for some breakfast and to dive in to some other part of the code.
_________________
Mike Kasprzak
'eh whatever. I used to make AAA and Indie games | Ludum Dare | Blog | Tweetar
View user's profile Send private message
Sirocco
Moderator

Joined: 19 Aug 2005
Posts: 9413
Location: Not Finland
PostPosted: Thu Jun 03, 2010 3:41 am    Post subject: Reply with quote

Quote:

#define all the way. You are right that it can make code kind of messy, but, I also find it worth it.


Too true. I don't care for the way it looks, but the convenience is too great to ignore.
View user's profile Send private message Visit poster's website
PoV
Moderator

Joined: 21 Aug 2005
Posts: 10792
Location: Canadia
PostPosted: Thu Jun 03, 2010 4:00 am    Post subject: Reply with quote

June 3rd - 8:00 AM
Smiles [Windows, SDL, OpenGL1] in back up and running in the new layout.

Found a bug in the timer structure above. It should read this instead:
Code:
// -------------------------------------------------------------------------- //
#include "../Timer.h"
// -------------------------------------------------------------------------- //
#ifdef USES_UNIX_TIMER
// -------------------------------------------------------------------------- //
// ... code goes here ... //
// -------------------------------------------------------------------------- //
#endif // USES_UNIX_TIMER //
// -------------------------------------------------------------------------- //

The difference is that the "../Timer.h" include goes before the #ifdef. Otherwise, the symbols implied by the includes wouldn't resolve correctly.

That technically means each file is doing a little bit more work inside the compiler, but I don't really care... heh. ;)


I cheated my way to getting OpenGL1 support up and running by moving my OpenGL1 implementation directly in to /GEL/Graphics. I figure I can clean that up, and include the variation code thereafter, since I think I'm going to need to structure the /OpenGL1, /OpenGL2 and such folders differently anyways.

Now, what to do next...

- - - -

June 3rd - 8:21 AM
I have a library called "Rebel"... or at least I did. It was a collection of utilities without a home. Those included:

- Console logging (Log.h) and Assert (Assert.h)
- Sequence (Sequence.h), which I think I've written about here once before.
- Bits (Bits.h) and Binary (Binary.h). Header files with constants for writing binary numbers an "b11010", and bit-fields as "bit10".
- Some other misc Utilities.

Rebel has been around for ages as my collection of those things, but the name really doesn't make a lot of sense anymore. So I'm retiring it finally. Log and Assert will go in to a GEL/Debug folder, and the rest under GEL/Util.
_________________
Mike Kasprzak
'eh whatever. I used to make AAA and Indie games | Ludum Dare | Blog | Tweetar
View user's profile Send private message
PoV
Moderator

Joined: 21 Aug 2005
Posts: 10792
Location: Canadia
PostPosted: Thu Jun 03, 2010 1:25 pm    Post subject: Reply with quote

June 3rd - 5:25 PM
On the Netbook.

I want to rework some aspects of my "Data" library (formerly living in /libc, now /GEL/Core), so a good place to start would be to see what I actually have.

The Data Library is how I do all my file reading and memory allocations. Things like decompressing a chunk of memory, hashing, or reading directly from disk by filename are considered standard features. An overview of each of the components:

- - - - -

Data.h - Arbitrary data (char) arrays. Wrappers for memcpy, memset, memcmp, allocating memory off the heap, etc.
DataBlock.h - Preferred version. Contains the size of the block of memory, simplifying many calls.
Array.h - Like DataBlock, but typed (int, float, whatever).
Heap.h - Actually, it's a stack (not a tree). An array of indexes that offset in to a chunk of memory, size of each assumed to be the difference in indexes. It doesn't currently support pushing and popping, but it could.
File.h - Wrappers for FileIO code (C FILE*'s). I have new plans for this.
Directory.h - Contains a "heap" of file names, a complete directory tree. Stored in a "heap" type so it can be bulk saved/read from disk.
String.h - Wrappers for C string functions. strlen, strcpy, strcmp, etc.
VFile.h - Read/Write to a chunk of memory as you would a file.

Endian.h - Utilities for handling endian change.
MD5Hash.h and SHA1Hash.h - wrappers for the respected hash algorithms (external).
Archive.h - Unfinished. A "heap" of data and a "heap" of file names.

StaticArray.h - Like Array, but allocated on the Stack instead of the Heap. Very nice with GCC, but MSVC just can't handle its use of the language (requires hardcoding).
Stream.h - Streaming interface. Fat and unused. Likely to be obsoleted.
Deque.h - Stub. Was to be a type that supported front and back pushes and pops.
StaticDeque.h - Stub
KeyArray.h - Stub. Was to be a typed Heap.

- - - - -

Data, DataBlock and Array all work great, and I'm very happy with them. No real plans to make any changes here, other than perhaps removing some fat (unused functions).

Heap, as the notes say, is actually a Stack. So I think I might want to tweak it so it supports common stack operations like pushing and popping. It's an associative type. An array of indexes and an array of data, so it's used to specify/identify several noteworthy chunks inside of a block of data.

File I have big changes planned. Right now it simply wraps the fopen family of functions (even returns a FILE*). I'd like to replace this with the idea of data sources, specifically, 2 sources. Content and Storage.

Content - A "content data source" is what represents game content files. Art, Sound, Maps, Meshes, etc. In games, the most common file access you do is reading content files. Most platforms do this with fopen, accessing the local file system with special paths (install directory, CD drive, etc). Something like NACL however has to pull its content from a remote server (via URL's). The idea of the "content data source" is that it doesn't care where the data comes from, when I ask for "GoblinTexture.png", I as the implementation go fetch it from wherever my content comes from. 99% of the time, all I will ever want to do with content is read it. However if I have integrated content creation tools, I may still want to write it, but that is the only case where Content would ever be written. You will never write game-state save data to Content.

Storage - On the other hand, you will write your game-state save data to Storage. That's the difference (and point) of the two. On PC's that's in a special place inside your home directory (per user account). On consoles that's often the memory card. With NACL, it's a sandboxed directory on the local PC. Files from storage are always both written and read.

... to be continued... my brother just showed up... heh. I'm totally not done talking though/making my file plans.
_________________
Mike Kasprzak
'eh whatever. I used to make AAA and Indie games | Ludum Dare | Blog | Tweetar
View user's profile Send private message
PoV
Moderator

Joined: 21 Aug 2005
Posts: 10792
Location: Canadia
PostPosted: Thu Jun 03, 2010 4:02 pm    Post subject: Reply with quote

June 3rd - 8:02 PM
Continued...

I'll come back to Content and Storage, since the rest of the list isn't too intense.

Directory works, so not much plans to change this. Given a folder it recursively searches it for all the files, and builds the list. I also have code in place for saving and loading completely populated lists of files, but I'll have to update it to suit my changes for File.

String is String. I can't think of anything that needs to change here, other than reading/writing files.

VFile is something I like, but haven't used yet. Really, I should be using a VFile for any cases where I write a bunch of random data to disk. That way, I could compress, munge, or hash/checksum the file before writing. Also it would be nice if it was written in a way that it was compatible with the new file interfaces.

Endian, I wrote it but I haven't needed it. I should double check how it works... heh. Actually it might be in my type writing code already, but I can't remember.

Archive, I didn't even realize I had this. I must have been getting started on the Android port, where I expected to need a way of storing and indexing data in an archive/wad like file. I can't think of any immediate needs for it, but it might be nice to have done and available as a virtual file system.

Stream is something I should be obsoleting. Streams were going to be how you read from URL's, or any future data source, so it's simple a placeholder container/template for future features. Where they would be useful is if they had something to do with packeting (internet downloads, or streaming video like files), but I think I may end up integrating streaming (or rather, the roots of streaming) in to the regular file reader. I'll deal with that later.

StaticArray just isn't practical anymore, since I can't rely on having GCC on every platform (as MSVC handles stack differently).

The rest (Deque, StaticDeque, KeyArray), just don't matter. Consider them gone.

- - - -

So, Content and Storage are what I want to call the 2 main places you will be working with files. On most platforms they will be handled by ordinary fopen family of commands, combined with platform specific ways of resolving the directories. And some platforms may require an outright different way of acquiring/reading/writing files (URL's, TCP requests, or just different function calls for accessing different devices).

The normal way I read a file right now is usually:
Code:
DataBlock* MyFile = new_DataBlock( "AwesomeThing.bin" );

I like the simplicity of that syntax.

Now unfortunately, the paths like those above don't resolve to content or save directories. Instead I need to build a path for each, and feed it to that function. Something like this:
Code:
DataBlock* MyFile = new_DataBlock( "/Res/Content/AwesomeArtThing.bin" );

That's worked for most platforms up until now, but if I'm introducing the concepts of Content and Storage directories anyways, I may as well resolve this stuff inside the calls. So the actual file calls should, given a path, figure out the correct path to the file given the platform (/Res/file.bin on Bada, for example), and feed that to the low level code for opening a file (the fopen wrapper).

I'm undecided though, but I think I may want to explicitly do the following for the data sources.
Code:
DataBlock* MyFile = new_content_DataBlock( "AwesomeThing.bin" );
DataBlock* MySaveFile = new_storage_DataBlock( "AwesomeSave.bin" );

Those then resolve to the correct locations give the platforms.

Up until now though, I was thinking I'd want the following to act as a synonym for new_content_DataBlock:
Code:
DataBlock* MyFile = new_DataBlock( "AwesomeThing.bin" );

But I'm asking myself, how do I then read/write anywhere I want on the file system? In a game I never would, but I think I'd still want it. So I'm planning an maintaining the original behavior (directly pass the path to content's chosen system, which will almost always be the fopen wrapper). I can't see (rationalize) a reason for direct access to the storage system though, just content.

It's back to the USES_GEL concept from earlier. If using a full-on GEL supported system, I'm going to want the abstraction of storage and content directories. But what if I'm writing a throwaway junk tool? "MyTool.exe InFile OutFile". Well if I force my basic code to imply a content path, I'm practically stuck writing fopen code manually, since my library isn't flexible enough.

With that in mind, I'd like both Content and Storage to be separately configurable per platform, path munger (/Res/File or http://myserver.com/File), and means of interpreting the path (fopen or some url curl'r).


So far I've written a quick one currently named FileHandle. It contains all the basic functions for opening/closing, getting the size, and reading/writing a file, all with fopen. The type "FileHandle" itself is a synonym for "FILE". That is then wrapped by a pair of fake types ContentHandle and StorageHandle, both currently implemented as #defines instead of inline functions. I think #defines are better here because of a variation I'd like to have, the read-only ContentHandle. In this case, when any of the writing functions are called, the macro resolves as an #error preprocessor symbol, causing the compiler to you're doing something bad. Just something I'd rather catch at compile time than run time.

Finally, the function inside the datablock (and other) libraries will call the appropriate munger, then the appropriate macro version. That's, in a nutshell, what I'm thinking here. I'm a bit tired now to do any more tonight, but that's what I'm thinking here.


Other things I'd like to see in GEL/Core:
- Streaming "structurally" built in to the data source code. Not the thread creation part, but the notification essentials needed to say "hey, load this and this. is the first this done yet?". Then have a higher level content streaming/queue/prioritization system living on top of this (inside Foundation, not GEL). These art assets are closer to me, so make sure they get loaded first.
- Some helpers for creating/loading files with "parts" in it. I have a texture format I'd like to write, that compresses all the textures and mipmaps in one file, but the top 1 or 2 mipmap levels are compressed separately. That way on low res cellphones like iPhone, I can ignore the really highres textures I can't actually display, and not get the performance hit of decompressing a larger file. Then on something like the iPad I can decompress and use them all, all from the same binary. iPhoneOS, Android, and even PC/Netbooks I see this being something I want.
- I think I'm going to rewrite my "Whitespace Tokenizer" code, since it currently relies on C++ StringStreams. Not sure if the new one will go here, or in utility (too tired to decide).

Well I've reached the end of my brainpower for the night. I realize it's early, but I woke up too early. I'll mull over this through the night, and see if there's anything I want to add in the morning (or do it all).
_________________
Mike Kasprzak
'eh whatever. I used to make AAA and Indie games | Ludum Dare | Blog | Tweetar
View user's profile Send private message
Gil
Developer

Joined: 14 Nov 2005
Posts: 2341
Location: Belgium
PostPosted: Thu Jun 03, 2010 5:31 pm    Post subject: Reply with quote

I just wanted to mention that I read all of that and I think it's really cool.

I have something similar partway done and you're encouraging me to finish it. Unfortunately, I've been stuck on a fixed point module for months now. I'm just really undecisive about it.
_________________
PoV: I had to wear pants today. Fo shame!
View user's profile Send private message Visit poster's website
Reply to topic GDR Forum Index -> Game Developer's Refuge -> Development Log - GEL and Foundation (Engine Tools) Page 1, 2, 3, 4  Next

Use this link to get a Sign-On Bonus when you get started!

All trademarks and copyrights on this page are owned by their respective owners. All comments owned by their respective posters.
phpBB code © 2001, 2005 phpBB Group. Other message board code © Kevin Reems.