GDR Forum Index
Podcast Podcast
Dev Dev Logs
Search Search
RSS RSS
Register Register
Log in Log in
Reply to topic GDR Forum Index -> Game Developer's Refuge -> Development Log - PoV's HTML5 PARTY!!! Page Previous  1, 2, 3
View previous topic :: View next topic  
Author Message
0xDB
Developer

Joined: 26 Dec 2005
Posts: 1651
Location: Your consciousness.
PostPosted: Sat Aug 23, 2014 1:17 am    Post subject: Reply with quote

Heh, my eyes read those fangs as little stumpy legs. ^^
_________________
0xDB
View user's profile Send private message Visit poster's website
PoV
Moderator

Joined: 21 Aug 2005
Posts: 10781
Location: Canadia
PostPosted: Sat Aug 23, 2014 8:09 pm    Post subject: Reply with quote

Ha, I never thought of that. :). You're totally right.

Hmm. Legs might be more interesting actually. The iris could always be a hidden mouth or something. :)
_________________
Mike Kasprzak
'eh whatever. I used to make AAA and Indie games | Ludum Dare | Blog | Tweetar
View user's profile Send private message
Sirocco
Moderator

Joined: 19 Aug 2005
Posts: 9411
Location: Not Finland
PostPosted: Sun Aug 24, 2014 7:11 am    Post subject: Reply with quote

Also, they could be arms hanging at his side while he floats, like a gazer/beholder.
_________________
NoOP / Reyn Time -- The $ is screwing everyone these days. (0xDB)
View user's profile Send private message Visit poster's website
PoV
Moderator

Joined: 21 Aug 2005
Posts: 10781
Location: Canadia
PostPosted: Sun Aug 24, 2014 11:31 am    Post subject: Reply with quote

Just an information dump. COOKIES!

Apparently one should store no more than 4093 bytes of data in a cookie.

Quote:
Chrome & Firefox - No limit to the number of cookies; limit of 4096 bytes per cookie.

IE8-10 - 5117 characters per cookie; limit of 10234 characters.

Safari on Mac - 4093 bytes per cookie. Limit of 4093 bytes.

Safari Mobile - 4093 bytes per cookie. Limit of 4093 bytes.

Android - 4093 bytes per cookie. No limit.

http://stackoverflow.com/questions/10570347/how-much-data-can-i-store-in-a-cookie

Main thing I'm trying to check on is ideal and widely compatible ways of persistent storage in the browser. Starting with cookies, then I'll look at that other thing.

http://stackoverflow.com/questions/1617250/storing-persistent-data-in-browser

EDIT: Web Storage is better, and widely supported.

http://caniuse.com/#feat=namevalue-storage

EDIT2: More to storange than I thought. sessionStorage and localStorage. Good good.

https://developer.mozilla.org/en/docs/Web/Guide/API/DOM/Storage

Quote:
1. In iOS 5 & 6 localStorage data is stored in a location that may occasionally be cleared out by the OS.
2. In private browsing mode, Safari, iOS Safari and the Android browsers do not support setting localStorage.


And it seems a straightforward way of detecting private broswing mode is doing exactly that: trying to set localStorage.

http://stackoverflow.com/a/17741714

EDIT3: The magic number, keep your local and session storage under 2.5 MB. That's reasonable.

Code:
Browser        LocalStorage         SessionStorage
-------        ------------         --------------
Chrome             2.5M                  2.5M
Firefox            5M                    Unlimited
IE10               4.75M                 4.75M


http://stackoverflow.com/questions/15840976/how-large-is-html5-session-storage
http://dev-test.nemikor.com/web-storage/support-test/

Quote:
1. IE 9 has a problem with numeric indexes and bracket notation.
2. IE 9 doesn't report quota errors using bracket notation.


Something worth knowing for LD.
_________________
Mike Kasprzak
'eh whatever. I used to make AAA and Indie games | Ludum Dare | Blog | Tweetar
View user's profile Send private message
Gil
Developer

Joined: 14 Nov 2005
Posts: 2341
Location: Belgium
PostPosted: Thu Aug 28, 2014 11:57 am    Post subject: Reply with quote

There's also Web SQL Databases, but support is patchy due to W3C dropping the spec:
http://html5doctor.com/introducing-web-sql-databases/

Pretty cool technology though. SQLite + Javascript = <3

Quote:
1. IE 9 has a problem with numeric indexes and bracket notation.
2. IE 9 doesn't report quota errors using bracket notation.


Ah, didn't know that one, thanks
_________________
PoV: I had to wear pants today. Fo shame!
View user's profile Send private message Visit poster's website
PoV
Moderator

Joined: 21 Aug 2005
Posts: 10781
Location: Canadia
PostPosted: Thu Aug 28, 2014 6:06 pm    Post subject: Reply with quote

Yeah I think something I wanted to use for my library code makes my IE minimum IE10, but I've been thinking a bunch lately about doing some serious work on the Ludum Dare website. New everything basically, and to make the site more responsive, lots of caching (both client and server), do more actual work client side. So who'dathunk, this homework might be actually be useful. ;)

I haven't had much to add lately, as most of my digging has been in to PHP'isms. Trying to find the best, highest performance ways to do things with PHP. Unfortunately, this can be tough as PHP documentation never deprecates things, unlike the W3C and JavaScript guys. I've never seriously studied PHP, but I need to now. :)
_________________
Mike Kasprzak
'eh whatever. I used to make AAA and Indie games | Ludum Dare | Blog | Tweetar
View user's profile Send private message
0xDB
Developer

Joined: 26 Dec 2005
Posts: 1651
Location: Your consciousness.
PostPosted: Thu Aug 28, 2014 9:49 pm    Post subject: Reply with quote

Sirocco wrote:
Also, they could be arms hanging at his side while he floats, like a gazer/beholder.
They're his GPS (general purpose stumps).

on the topic of local storage/cookies/session stuff:
I think I would simply store all state directly inside the client-side script and for tool output like from map-editors or even for save-games I'll just have classic save/load mechanisms which let the user pick a file-name to save-to/load-from. For application settings though, seamless local storage would be useful, so users would not have to pick a settings file for loading each time they run the application. Another, more expensive(as in $$$) option, would be to require users to register/login and store individual settings in a database server-side.
_________________
0xDB
View user's profile Send private message Visit poster's website
PoV
Moderator

Joined: 21 Aug 2005
Posts: 10781
Location: Canadia
PostPosted: Fri Aug 29, 2014 11:00 am    Post subject: Reply with quote

Google is supposed to offer free storage for everyone in their game APIs. Generally speaking, we can store 2.5 MB of data directly in the browser's localStorage (and sessionStorage, each). I was about to make a comment on "how about persistent storage across devices running the same broswer", and had a duh moment "uh, that's what cloud storage is dude". ;)

EDIT: https://developers.google.com/games/services/common/concepts/savedgames

EDIT2: Sounds like things are changing a little bit. https://developer.android.com/google/gcs/index.html
_________________
Mike Kasprzak
'eh whatever. I used to make AAA and Indie games | Ludum Dare | Blog | Tweetar
View user's profile Send private message
PoV
Moderator

Joined: 21 Aug 2005
Posts: 10781
Location: Canadia
PostPosted: Fri Aug 29, 2014 10:49 pm    Post subject: Reply with quote

So I have been doing a lot of research. Servers, databases, server-side scripting, etc. It is kind of exhausting really. I'm going to summarize some of the things I've found. It's really kind of crazy how much reading I've had to do for such straightforward answers. Well I guess that's the thing, they really weren't straightforward at all. Going to repeat it mainly just to test my comprehension.

- Web developers love their acronyms.
- LAMP (Linux+Apache+MySQL+PHP) is the norm that most servers run. Reliable, but Apache can be greedy with threads and memory/CPU usage.
- LEMP (Linux+"E"nginx+MariaDB+PHP) or LNMP (same) is *I THINK* better. Nginx is a very fast and efficient web server, perhaps lacking a few oddball features, but is your best bang for buck on CPU and RAM usage. Fair to say its #2 on the web next to Apache, and not because its any worse. LEMP is typically MySQL too, but...
- MariaDB is a fork of MySQL. Remember when Oracle bought Sun and ruined Java and OpenOffice? Well they own MySQL too, so MariaDB was created to basically cut them out of the loop, same as LibreOffice. Compatible up until 5.5, but is now 10.x, to basically say it won't necessarily support MySQL 5.6. Functionally identical to MySQL, even the API is still mysql_blah. Adoption is still early, but Google, Mozilla and Wikipedia use it now instead of MySQL.
- Every character in that acronym is pluggable. Some server guys swear by FreeBSD over Linux.
- Perl and Python are options instead of PHP, not to mention there are now alternative PHP implementations like Facebook's HHVM/Hack. I've read conflicting things about HHVM, so it's only on my radar as a way of confirming a PHP API is worth using, as many are unavailable in it. PHP has a strange documentation that combines builtins and popular libraries. Only thing, those libraries never get deprecated, at least in the documentation. :P
- The fastest/most compatible way to share information between PHP script executions is with a library called APC.
- As of PHP 5.5, APC has been deprecated. However, that was because APC used to be both a data sharing library and a PHP script caching library. Caching has been replaced by OpCache as of 5.5, and the sharing library is now known as APCu. It's is functionally identical, same code and everything. For older PHP versions, you can still use APC.
- That said, if you plan to reuse data within the same script execution, APC is slower than local variables. Of course, local variables don't persist across executions. For best results, store the data given to you by apc_fetch, and use the local copy. If the APC use is one shot, then just use it.
- An alternative to APC is MemcacheD. This is an external service you run outside PHP, and talk to via TCP. If everything of yours runs on a single server, you shouldn't even bother with this. If you run multiple servers, this can be used for APC like behavior across those many machines. Again fi youre one machine, APC will beat it hands down.
- OpCache looks like something that can be just enabled. Then, all scripts will be cached precompiled in memory after their first use. There is a library available on the PHP side to find out how well the page caching is working.
- PHP is rather filesystem savvy. It could be used to read and write files from the local disk, as well as get attributes of files. This can be especially useful when creating a caching solution. Just look at file time, and you know.
- While it does depend on a number of factors, retrieving data from a MySQL database is probably the slowest of the mentioned methods of getting data. That said, it is extremely flexible and consistent. At least, if things are built correctly. Some of the data will be cached in RAM, so requests for it will respond very quickly. Like MemcacheD, it's a separate running service communicated to via TCP. Of course, a database will store a hard copy of its data on disk, so it is persistent and reliable unlink MemcacheD or APC. The key bottlenecks are TCP, and badly constructed queries.
- Also as of PHP 5.5, the mysql_blah family of functions are deprecated. They will still work, but raise an error warning you to upgrade.
- the new DB libraries are MySQLi an PDO. The former only supports MySQL databases (I.e. MySQL and MariaDB), where as PDO supports everything else via drivers.
- there is a slight performance advantage to MySQLi over PDO. PDO may have a better syntax, but both libraries aside from what back ends they support are functionally identical.

That's all I can think of at this hour.
_________________
Mike Kasprzak
'eh whatever. I used to make AAA and Indie games | Ludum Dare | Blog | Tweetar
View user's profile Send private message
Adam C. Clifton
Contributor

Joined: 04 Nov 2005
Posts: 221

PostPosted: Wed Sep 03, 2014 9:28 am    Post subject: Reply with quote

Ooooh a topic I actually know about!

Your LEMP stack is what we eventually evolved to using at Firemint/Firemonkeys. A long way from the IIS6 I originally setup.
If you are super serious about performance, there's XtraDB, a faster version of MariaDB. They're all drop in replacements for Mysql, but most people go with Maria these days because boo Oracle.

It's amazing what you can do with memcache and good enough database access code with lazy db connections. We had a set-up that would load the user data from the database, then store it in memcache for 30 minutes or so, then the next time the player synced, we'd pull the data from memcache instead of the database. If modifying the data, write it back to memcached and the database. We ended up with more than half the connections not even touching the database.

These days I'd probably go with redis as my cache as you can do more interesting things and store more data.

On my own projects I use sqllite which has some gotchas of it's own, but it's ACID and I don't get enough traffic to even think about optimizations, and even then I'd go for a cache before a 'real' database.



There's a lot of hate on PHP but I don't mind it. I think most of it comes from people learning PHP as a first language and writing crap, since you're a real programmer I don't think that will be an issue. Opcode caches make it pretty close to native C++ in most benchmarks I see.

Apart from the triple equals operator, my other favourite bit of weirdness about PHP is operator precedence:
if ( $flags & MASK == 1) do_something();
Does not evaluate as it would in C.






Obligatory XKCD, http://xkcd.com/908/
View user's profile Send private message Visit poster's website
PoV
Moderator

Joined: 21 Aug 2005
Posts: 10781
Location: Canadia
PostPosted: Wed Sep 03, 2014 5:00 pm    Post subject: Reply with quote

Oh nice. I didn't know you were at Firemint. I'm sure I've met one of your bosses way back when making iPhone games was new and exciting. ;)

Thanks for the note about XtraDB (Percona). I've been avoiding them since I assumed their stuff would be paid, just as Nginx and Zend have paid stuff and the o-sauce project is the free one.

I think I still like APC (APCu) since having shared memory sounds like something that should be built in to PHP. Redis sounds interesting though, not that I particularly know what makes it better an Memcached, but everyone says its faster so :D.

Mainly though, I get the impression PHP has a shitty optimizer. What I mean, in C/C++ land, you could wrap a function with another one and not incur function call overhead for pretty much the past 15+ years. What I would love to do is support multiple systems, use APC(u) until I really need to change (may not), then and only then run such a server.

Another person suggested Laravel, a PHP library. It looks decent, but I do worry about how optimal PHP really is. Looking at benchmarks of MySQLi and PDO (and even legacy MySQL lib) really bothers me, as PDO should effectively be just a wrapper. So I search far and wide for all the performance know-how I can find.

I haven't yet run in to the PHP === operator. The JavaScript one I have, and it's fine. It makes sense. I use it. I'm not surprised to hear & has a wonky operator precedence. I never trust bit ops anyway and always wrap them in ()'s... Which can be often since I do like to shift :).

PHP seems fine. I'm less bothered by it now than I used to be. I do wish it was more C-like though ($'s, *shrug*). Part of me wants to use node.js instead of PHP, but I honestly just don't trust it enough yet. PHP is proven and reliable. I've run it for on my own servers for over a decade.

Spent the evening the other day reading the nginx documentation. I really like it :). It's to the point, where Apache always felt monolithic and insane. Mind you I've never seriously given it the deep read, but I've run it on ever server of mine until now, and I know what it looks like when it chokes. :). I need to both build this website and run this server, so if I can diagnose problems faster because there's a shorter list of features, that's a win for me. Honestly, I can't even think of anything it's missing. I could wrap a node.js server, or any server for that matter, and make it serve all the static files with just a few lines. Wonderful. Working on knowing my stack. :)


Anyway thanks. Feel free to throw more tips at me. I'm trying to assimilate as much LEMP wisdom as I can. I'm about to embark on quite an outrageous project (rebuilding Ludum Dare), and I want to make sure I'm making all the right decisions from the beginning.
_________________
Mike Kasprzak
'eh whatever. I used to make AAA and Indie games | Ludum Dare | Blog | Tweetar
View user's profile Send private message
PoV
Moderator

Joined: 21 Aug 2005
Posts: 10781
Location: Canadia
PostPosted: Fri Oct 10, 2014 11:44 pm    Post subject: Reply with quote

Been working on Ludum Dare stuff, which mainly means PHP+MySQL. Nothing really to show yet, but I want to nerd out a bit.

Been working with 3 kinds of Data:

1. Finances (Paypal and Patreon donations)
2. Steam Curator and Group Info
3. Twitch and Hitbox Broadcasting Info

The data is available in a variety of forms.

- & delimited HTTP Post Response (Paypal)
- CSV files (Patreon)
- XML Response (Steam Group)
- HTML Response (Steam Curator)
- AJAX+HTML Response (Steam Curator)
- JSON Response (Twitch and Hitbox)

I don't deal with it yet, but Reddit data is also available as a JSON response. I've also been thinking about what I could do to get IRC channel info (run a bot?).

Anyway, the point is that I had to crash course myself in practically every way of retrieving data as PHP. I've only just started writing the broadcast data to a database table, but I've loaded and parsed everything. I roughed out the table design for the rest of the data, but I wanted to do my initial experimentation with less critical data.

The Paypal data already exists. I started with a general donation plug-in written for WordPress, and enhanced it a few years ago (added an expenses table, practically the same as the donations table). I hadn't touched the code since until recently, when I finally fixed some bugs: 1. Donations couldn't have decimals (easy fix). 2. Wasn't recording fees or the actual email address (just a user entered one). 3. Automatically adding donations to database after Paypal confirms the transaction (did not work).

Again, the rest of the data is collected, but I'm not storing it yet... With one exception.

Broadcaster data. Both Twitch.tv and Hitbox.tv. I looked at a few more streaming services (MLG, Azubu), but they're both invite only, which is uncool.

I've set up a Cron Job that calls a PHP script whose job is to, every 5 minutes, see whose online (broadcasting live video). If they're online, give them 5 points (1 point per minute). This is a simple database of user info. The service User IDs plus an ID representing the service is the combined primary key, but I'm also indexing the timestamp. This will be useful to know whose online, and who our most active broadcasters are.

In the same Cron Job, I have another lightweight table I'm logging the current broadcast stream counts, and viewers of all streams to. So every 5 minutes, 38 bytes of data plus indexes are written, times two services. I wanted this finished tonight, so I could have some real data in the morning. This data is for watching daily trends in streamers+viewers, and to let me know how close we are to 24 hours of constant gamedev streaming (and once Ludum Dare 31 rolls around, how long a streak can we keep up). :)

Trying to make the most of the data. Tomorrow I'll have something real to look at, and I can start digging into more optimization. I'd like to see how effective the SQL queries are. I think I optimized them well, but I will be sure until I have data.

And then there's charting. The data is only so useful, until it can be displayed.

More data fun tomorrow, before Canadian Thanksgiving with the family.
_________________
Mike Kasprzak
'eh whatever. I used to make AAA and Indie games | Ludum Dare | Blog | Tweetar
View user's profile Send private message
Skywise
Member

Joined: 18 Dec 2005
Posts: 75

PostPosted: Fri Dec 26, 2014 11:44 am    Post subject: Reply with quote

Haven't read the whole thread (just recent posts), but I want to throw in that for Web Apps, NoSQL is a great solution. I particularly like document DBs where you can just stick JSON in, get JSON back out. MongoDB is arguably the gold standard in that category right now; CouchDB is what I've used more, because it comes with a built-in RESTful database.

My first reaction to NoSQL DBs was "MySQL is fine; sure, the data I'm storing isn't really table-like, but some middleware will handle the conversion for me anyway." This is coming from a background in Java, where adding Hibernate annotations to a ton of classes in your persistence layer made you say "wow, that was easy."

With JavaScript+CouchDB, if I want to persist state, I JSONify it and make an HTTP call. It's almost transparent, and importantly, I'm not doing any kind of manipulation of my data in order to interface with my persistence store. I don't need an ORM or some intervening library to do that; I don't even need an adapter library for my database, because it's just a RESTful interface that's dead simple to talk to from code.

Part of my resistance to NoSQL had been the "what's better about NoSQL" question. In terms of capability, tabular relational databases can express the same information, and libraries can make them pretty easy to deal with. But if you flip the question around and ask "why use MySQL?" then the NoSQL solutions make sense as a default.

I would want to specifically use MySQL (or similar) if:
1) I needed to perform some reasonably complex database operations/queries that will be more efficient on an indexed, tabular, relational database.
2) I don't anticipate a need to "scale out" and distribute my database across multiple hosts.

These reasons are a little contradictory, though; (1) doesn't generally matter on smaller scales (e.g. if you just need to store and load save game state), but if you reach a certain scale you bump into (2). There's a goldilocks zone where MySQL makes sense; that region matched well with web site usage patterns around ten years ago, and there weren't a ton of other options, so MySQL became a reasonable default. The situation's just different now.
View user's profile Send private message AIM Address
Reply to topic GDR Forum Index -> Game Developer's Refuge -> Development Log - PoV's HTML5 PARTY!!! Page Previous  1, 2, 3

Use this link to get a Sign-On Bonus when you get started!

All trademarks and copyrights on this page are owned by their respective owners. All comments owned by their respective posters.
phpBB code © 2001, 2005 phpBB Group. Other message board code © Kevin Reems.