Case Study: TCG in React

I work on projects through a process where I am constantly tearing down walls, but I leave talking about it to the hypothetical "finished point". That's a mistake.

I'm currently working on two projects, Lanarts and a game I will tentatively call Ciribots, a card game. I have talked enough about Lanarts for now, let's talk some about Ciribots. I'm still lagging on open sourcing it, but that's as much due to shy privacy as any vague hopes of monetization.

"Ciribots" new prototype, with animated React glory.

Work on Ciribots currently falls under 3 categories:

• Initial game design: While I've been mulling over the rules of the game for a long time now as the game was played in paper play. Currently the firmly decided rules are, roughly:
• Cards make or interact with pieces on the board.
• Pieces move like chess queens, but only up the their movement value in squares away.
• Pieces can attack whoever a queen could attack, up their range value in squares away. Pieces don't move to the enemy square ever, unlike chess.
• Cards have a health value that resets each turn. It must be brought to 0 in a single turn for the unit to die.
• You must move a unit to one of the two centre squares on the other side of the board to win.
• You can move with 2 distinct units in a turn. All units have energy to do one action each turn (like MTG tapping), which is used to attack, deal damage back when attacked, or to do a special action printed on the card. 
• Board game engine that lends to easy card changes: This is probably the most interesting part of the implementation too me, although also a part I wish could be magically written. Writing in C++ for performance, compiling with emscripten for use on front end. The board game engine is the backbone of the rules enforcement, and also any AI players. It is being written with both absolute performance (e.g. roll back states in AI to avoid any full state copies, keep exactly one state object) and extensibiliy (incorporate any new cards) in mind. It's my favourite kind of tricky challenge - designing a performant system that might be extended to do *anything*.
• UI in React: The game is a react app. I'm on the JSX bandwagon. Probably to the point that I'm not really interested in non-React front end stuff. Too many goodies for language nerds like me with TypeScript+React being both static and declarative. My most recent rewrite in React-as-of-May-2018 focused on using CSS animations + React animations, which mostly work amazingly other than some jankiness. 

And this I shall publish, because I have to remember partial results are worthy to talk about.

-Adam

Lanarts Update October 15th, 2017

Download here!

Boy has it been a long time since I released. Releasing can be a very hard mindset - Lanarts always has some features that are more experimental than they should be. My friend Pat reminded me how sorely needed Lanarts releases are, so here goes (the last time I regularly released was in 2012!).

Greatly improved random world generation:

As I alluded in my last Lanarts development blog (it wasn't even a release), a major addition a while back was the random generation of the overworld that connects dungeons. This has been greatly enhanced to the point that the map has arguably more interesting features than when the map was hand-made. Notably, as with other rogue-likes such as Dungeon Crawl Stone Soup, hand-designed 'vaults' can be blended into otherwise procedurally generated rooms. In addition, many additional dungeons and bosses have been added to the game.

Lives mode:

Non-hardcore mode now has 100 lives per player, shared by all players in a lives pool. This makes death still relevant - you want to save the players who keep dying - but make the game feel much lower stakes, as a casual mode should. A previous iteration had the game only end when everyone died. This was exciting - however, it discouraged everyone working together. 

Split screen:

Play with a controller plugged in and automatically play with your friends! Currently the only way to set the classes of other players is by setting the L2, L3, etc environment variables, eg L2="White Mage". If you want the first player to play with a controller, use LANARTS_CONTROLLER=1. I understand this is a brutal interface, more work to follow!

Pixel lock is banished:
Pixel lock was the name given to a 'feature' of Lanarts where, due to enemies being considered solid objects, and the 'pushing' mechanism that the game provides you being quite limited, that some times you would get completely stuck and have to fight your way out. This created some of the tensest moments in the game - a seasoned player would fear corners greatly. However, it greatly limited design space! Suddenly the biggest worry about a group of enemies was not their combat power, but the fact that they would physically pin you as a fundamental part of the game. Now, you simply move half speed through enemies. The resulting game play feels intuitive, you don't notice the weird aspects of pushing enemies anymore, and there is much more room for big groups of enemies.

New game advancement structure:

Key items that unlock areas is a major theme going forward. Currently in the game, you must first get a blue key, then a yellow key, and then three 'Lanart's to progress through the game. Lanarts (which obviously exist to give the game's name a concrete reference) are currently objects with no effect other than being collectable, and found in major areas / when bosses die. Collect 3 lanarts, enter the final area, beat the final boss Pixulloch, and you win the game.

Missteps and not-missteps:
Why does Lanarts take so long to make? First and foremost, Lanarts is a learning project. I'm really not afraid to take my time and make mistakes. However, I should take more time to talk about my mistakes.

One mistake was trying to rewrite Lanarts in a new engine. I learned a lot about MOAI, hacked a lot to make it work how I wanted it to, but at the end of the day questioned whether I was really better off using a large engine that I don't understand. This isn't to say using MOAI wouldn't have immense benefit - but rewriting is such a costly task that you better be getting serious metaphorical cash for your endeavour. I saw potential but a lot of hard work ahead.

Now let's get into something I did right - the most crucial thing I did for Lanarts development I believe was to abandon the MOAI version. The key insight was that I started afresh with the MOAI version and did some really cool things because I had a fresh slate - but that didn't mean I couldn't do those things in the old code base! So, while the previous development blog promised a procedural overworld and a new stat system, these features were actually tied to what was essentially a mock-up of Lanarts.

Porting over the procedural overworld was a very important task - especially for my motivation. Before, my most exciting feature (greatly more sophisticated world/level generation) was tied with a version of my code that needed monumental work. When I ported over the code, I gained a newfound appreciation for my old code and I felt like working with it was well worth it. Some rounds of cleanup later and, while not ideal, I am very satisfied with the state of things. I can truly do a lot of things really nicely.

What about the stat system which I was excited about last blog? Turns out I was thinking about stat systems all wrong. When you see a stat system with a lot of distinctions, it can be exciting - but only as exciting as those distinctions actually mean in gameplay. One thing I've learned about Lanarts is that it doesn't work well as a subtle game. Stats need to be very clear in their impact. As such, I took a key component of the stat reformulation - damage types, and kept only that. To increase the clarity of damage types it was important to simplify the stat system. Now in Lanarts you have to worry about your power, which adds 5% of your base damage for every point, and their resistance, which subtracts 5% of base damage for every point, and what types are relevant. Previously, stats would also interact with your base damage and complex solutions were used to avoid degenerate cases. Overall a big win, and that leads me into my other feature.

Damage types, new classes:

Play as the Red Mage with strong spells of fire or the White Mage with lightning power! Necromancers largely pay with their own life for their devastating magic, and summon shadow creatures to aid them. Use ultimate spells to clear out large areas.

One-file C++ Utilties

Summary:

Here I show off self-contained utilties I have accumulated for C++ over time. Included are small functions for:

1. Directory manipulation and searching
2. Creating a timer
3. Simple performance stats
4. Printing the type of an object.
5. printf-like string manipulation

The name is slightly dishonest, but few-file C++ utilities didn't have the same ring.

Overview:

While working on lanarts I frequently had to write code that was simply an #ifdef between UNIX land and Windows land. I made sure to minimize the amount of #ifdef's inside the actual game engine, pushing it into small C++ classes or one-file modules

Time has passed and they have remained useful to me; I publish them here for anyone to use. I figure stand-alone C++ utilities, where you can simply paste one or two small files into your code are worth releasing. You can incorporate them into namespaces if the need is felt; I usually go global for convenience.

Note 1: If you find yourself hungering endlessly for classes like these, I heartily recommend Boost. Personally though, I like to make due if I can with smaller dependencies.

Note 2: Most of these are from C++03 usage, so if there's a better way to do it in C++11 I may have missed it.

1. Simple directory access

directory.h
directory.cpp

It uses tinydir.h:

tinydir.h

This works on Linux/Mac/UNIX-like systems and Windows.

Don't expect much options here, but it is quite complete. 'search_directory' provides recursive directory searching, retrieving a filename list. The name pattern takes wildcards.

'ensure_directory' can create a directory if one does not already exist, returning false on failure.

See the header file for details.

2. Simple timer utility

Timer.cpp
Timer.h

Ever messed about trying to have a simple timing class in C++?
This one relies on using gettimeofday on Linux/Mac/UNIX-like systems, and emulating it on Windows. The Windows code was from Stack Overflow.

Provides these member functions:

Timer():
   Sets a timer, get_microseconds() == 0 at this point.
void start():
   Resets a timer, makes get_microseconds() == 0 again
unsigned long get_microseconds():
   Time in microseconds, multiply by 10e-6 to get seconds. May want to change signature to 'long long' by editing the source.

Example usage:

Timer timer;
... stuff occurs ...
cout << "Seconds have passed: " << timer.get_microseconds() * 1e-6;


3. Simple performance stats

PerfTimer.h
PerfTimer.cpp
perf_timer.h

Simply, PerfTimer uses the above Timer class to create timers suitable for use with long-running methods (a map to the type name is used for convenience, but isn't super-fast).

Example usage:

Simply include perf_timer.h, and have:
PERF_TIMER();

at the start of your function. It will record simple statistics such as average time, max, standard deviation, etc.

Then, when you want to see the results, simply run:

perf_print_results();

And you will get fairly nicely formatted results. The advantage is this approach is enough to see obvious bottlenecks (ie, this function is taking 75% of the time) and does not require the use of external tools. 

Example Output:

func bool analyzer_follow_entity(AnalysisState&, int, double):
        >> total 3797.9561ms
        calls 2801
        average 1.3559ms
        max 23.2120ms
        std.dev +-1.8125ms, +-133.67%

4. Simple typename printing utility
typename.h
typename.cpp

Useful for templated functions, when you want to pretty-print an object's type. Note that proper use of a debugger may or may not be more appropriate; littering print statements still has its place, though.

Example usage:

template <typename T>
void my_template(const T& obj) {
       cout << "got object of type: " << cpp_type_name_no_namespace(obj);
}

5. 'sprintf' into an std::string

strformat.h
strformat.cpp

This provides a global function called 'format' (again, edit to your needs!) that takes printf-style arguments and returns an std::string. To me, this clears a pain-point where I prefer printf-style syntax for output, but cannot use it to make temporary strings in output functions.

Note that the code is not designed for efficiency for strings > 512 bytes, but it does guarantee correctness.

Example usage:

std::string my_str = format("(%d, %d)", 1, 2); // -> Goes to "(1, 2)"


What else?

I have a Lua wrapping library (C++03 compatible) very similar to luabind but with no dependencies, and quite small size. It may be worth talking about in a follow-up.

A Parameterized Attack on Programming

2018 EDIT: One of those posts you look back on and wonder what you were thinking. I think I know?

I will be frank; I don't like the programming language L. If I could stop people from using it directly I would, forcing them to first formulate their problem in a better language.

Practitioners are subtly but profoundly swayed to take shortcuts with correctness because of a lack of simple or idiomatic expression in L -- after which, of course, they are encouraged to compromise performance for some measure of the correctness gained back. Double-of-course -- correctness is everything!

Truthfully, there is little wrong with L, but more the culture surrounding it. By refusing to use anything but L, a mono-culture forms, and L is forced to handle more and more concept mappings that bring L to the very limits of (and, sometimes, off the charts of) its intended cost/benefit trade-off.

The economic viability of programming has given it a very dense and awe-inspiring history -- but correct language is essential -- all industries are doing what will inevitably be called 'early programming'.

Consider the culture of programming as a whole:

Money drives programming hours in a series of chaotic directions. Man-millenia become spent on redundant interfaces to obsolete versions of redundant programs by obsolete and redundant companies. Those observing these practices identify similar but frustratingly hard to generalize flaws over and over. It is discouragingly hard to incrementally improve something for which your system has no intelligence.

And of course, why bother with thinking about a gradual path of improvement when you face costly ad-hoc reformulation, creeping obsolescence, changing requirements, no market demand, et al staring at you. You can almost feel the boulder rolling back as you reach to push it up the hill. Individuals in these situations sometimes find solace in the delusion that since they are avoiding changes they are avoiding the problem of programming -- only to be pulled back in for one last 'hit' by the winds of change. And this describes our dense landscape of opportunity, where history is not only doomed to repeat itself but to synchronize at many points as industries struggle to solve problems in excitingly identical ways.

And that's precisely what's wrong with the programming language L, and it has almost nothing to do with the programming language L.

When our most fundamental set of tools are thought of, therefore spoken of, therefore implemented as constraints not strictly the minimum of our problem domain, we create (at least slightly) divergent paths of progress. As long as we continue to rely on competing ad-hoc imperative logic models to handle conceptually non-imperative tasks we have literally no hope for normalization (viz Halting Problem).

Only when we leave 'early programming' can the very confused non-software-oriented industries return to business as usual. Producing correct software is a minefield compared to ensuring correct manual labour. Once the designer for even a non-technical discipline can manipulate a set of sufficiently expressive symbols that appeal to their intuition without fear of being forced to reason about arbitrarily-imperative behaviour or any other ad-hoc idiosyncrasies, then we can say we are observing modern programming.

Lanarts, Devblog for Release 13Alpha

I keep wanting to make grand long blogs describing Lanarts development in-depth. That doesn't seem to be working; I'll try shorter blogs.

Lately, there's some exciting stuff happening with Lanarts, and it'd be a shame not to share -- especially because I don't know how long it will take to stabilize all the new features!

Randomly Generated Overworld:

This happened much sooner than I thought! For those of you following, Release 12 introduced an overworld -- however, it was not randomly generated.

I really didn't know how I'd approach the overworld -- connectivity is tricky with non-grid layouts. The approach taken was fairly simple. The algorithm is basically just a bunch of random polygons, and lines connect nearby polygons. I used libxmi to draw the polygons efficiently, and baked it into my map generation library. From there, Lua code controls the map generation, and it it is quite well-performing, even for large maps.

New Stat System:

From a gameplay perspective, this is huge (and criminally overdue). This replaces the previous simplistic stat system (Melee/magic damage types, and resistance/effectiveness/damage/defence for each). Now possible are elemental damage types, bonuses against specific monsters, easily-definable custom status-types, equipment that can grant virtually any ability, etc.

I could probably write a whole blog about the nuances of the stat system, but essentially it was rewritten in clean Lua with many interesting possibilities. The stat system comprises of base stats, and components of these base stats, eg the inventory, result in bonuses to the derived stats. This way eg you can have a spell that is granted only when wielding an item, since it can be added to the derived copy of the stats. This derived copy is computed whenever needed.

Roadmap:Finally, the goals I laid out for Lanarts Release 1.0 have started to take shape. The two issues being tackled by this release plagued me for a long time, and I'm happy to have found elegant solutions to both of them (thanks to the expressiveness of Lua!)

Another, relatively unseen, issue that I have gone a long way to fulfilling is the ad-hoc nature of the Lua API. Recently, a lot of effort has been taken to clean-up and modularize the Lua part of the code. The Lua API continues to become more lower-level and general to take full advantage of Lua's expressiveness.

While development for release 13 is well underway, there is still lots to do. Unfortunately, there is a lot of balancing work ahead with the new stat system, but I have some promising ideas for automating this task.

Feedback:

Please direct any bugs, comments, or inquiries either as a github issue, a comment on this blog, or through IRC or email:

IRC Channel: #lanarts at freenode
Contact Email: domuradical@gmail.com

Feedback very welcome !

Lanarts release 12, now with overworld!

New release, here is the download for Windows!
Source-code for this release: Source

This one was pretty delayed. It's about time I released a version with some of the large recent changes.

READ BEFORE PLAYING:

The game does not provide much instruction (the README is woefully outdated), but your mission currently is to clear the dungeon that has warning skulls near it. You begin near the first of two dungeons.

It is recommended you move with WASD, melee with H, fire spells with YUIOP, mouse controls for everything else. Mouse controls for spells are fairly suboptimal at the moment.

Additionally, 1 to 9 for inventory slots 1 through 9.
Right click is needed to drop items, however.

Screenshots and Changes

Major changes:

• There is now an overworld with multiple dungeons!
• As a result, almost all the areas in the game have been majorly revamped. The red dragon is now on the overworld, a different boss resides in the first dungeon.
• There is now a new logo, courtesy of Kevin Siapno.
• Two new spells - Minor Missile, a mana-conserving Mage spell, and Expedite, an archer spell that causes them to move much faster for a short time. 
• Poison Cloud has been moved from the Mage to the Archer. 
• Magic Blast is now a 3rd-level Mage spell.
• Minimap is much more visible. You can click to adjust zoom level, right click to look around.
• [Code-only] The Lua code has been refactored to be much more modular, and has grown significantly. The area generation code has been rewritten (and improved, in the process) in Lua.

Too many minor changes to list. If curious, you can always check the commit logs.

Special thanks to Matt Simon, Steven Trumble, putterson, and the 2013 Toronto Red Hat interns, for their help testing the game. Thanks as well to Kevin Siapno for the original logo.

Downloads and Discussion

Windows:
Download!

Linux:
Compiling on Linux is very easy!
Download the source here: Source
-or- if you want the very latest updates:
git clone http://github.com/ludamad/lanarts --depth 1

Then from the folder you extracted or cloned to:
./run.sh

This will build and then run the game. If you are missing dependencies on Linux (probably the case) please run either fedora-deps.sh for Fedora, or debian-deps.sh for Debian or Ubuntu.

I do not yet provide linux packages, please do kick me if they'd be useful for you.

Other systems:
The game has been written to be very cross-platform; however it has not yet been tried on Mac or other systems. Anyone willing to help try Lanarts on a different platform is asked graciously to email domuradical@gmail.com.

Feedback:

Please direct any bugs, comments, or inquiries either as a github issue, a comment on this blog, or through IRC or email:

IRC Channel: #lanarts at freenode
Contact Email: domuradical@gmail.com

Feedback very welcome !

A 'fork & exec' surprise

See my gist for a self-contained C++ example of the problem code:
https://gist.github.com/ludamad/5579602

So recently during my work at Red Hat I was looking into an issue with Java on Linux (with IcedTea7+ and Oracle's JDK).

The straight-forward seeming code

FileWriter writer = new FileWriter(someScriptName);
writer.write( ... some contents ...);
writer.close();
giveFileExecPrivileges(someScriptName); // eg, using UnixFilePermissions
Runtime.getRuntime().exec(someScriptName);

This code snippet creates a new text file, sets executable permissions, and then tries to execute it.

This code runs just fine when executed in a single thread. Introduce multiple threads (also running this spawning code) and every-so-often you will get an IOException with the message 'error=26 Text file busy'. This IOException means that Linux is refusing to execute the file because it is open for writing.

However, this is clearly the only place we ever open the file - and we promptly close it. So what's going on ?

An aside about executable text files

On Linux (and originally, UNIX), we can execute a text file by looking up the 'shebang' line, eg at the start of the file:

    #!/bin/bash

This tells the kernel which program to use to execute the file. You get 'text file busy' basically if you try to execute a file that is being written to. Normally this would be a text file (you can get this error with binary files as well, though).

Digging deeper

Unfortunately, you can stare at the Java code endlessly; it won't tell you anything. Luckily, we have OpenJDK. A quick foray into UNIXProcess_md.c shows the native implementation of Java's Runtime#exec method. It is implemented using the system calls 'fork' and 'exec'. (Actually, it uses 'vfork', in OpenJDK7, but it's not important.)

Fork, if you're not familiar, essentially creates a new copy of the process, with enough copy-on-write magic to do it efficiently. Worth noting, this child process gets a copy of all the file descriptors opened by the program.

To spawn the program we actually want, we then use 'exec'. This replaces the current process with the specified executable. However, we continue to inherit whatever file descriptors were already open in the new process.

Don't copy my file descriptors, please

Normal programs will take care that the file descriptors copies are closed upon 'exec'. This prevents unwanted information from leaking. Indeed, this is the case in Java-land, where all open file descriptors are marked with FD_CLOEXEC.

This solves a very important problem -- that file descriptors can leak into the child process. However, this does not prevent the copy from occurring in the first place!

Back to the code

Is this knowledge enough to pose an explanation of what's going on ? In fact, it is.

FileWriter writer = new FileWriter(someScriptName);
writer.write( ... some contents ...);
// Say another thread executes 'Runtime.getRuntime().exec(someOtherScriptName);'
// this forked process will have a copy of the file descriptor we are about to close!
writer.close();
giveFileExecPrivileges(someScriptName); // eg, using UnixFilePermissions
// It is very possible that the forked process has not released the copy here!
Runtime.getRuntime().exec(someScriptName);

Unfortunately, the fork from another thread can copy the file-descriptor, suddenly be set aside to let other threads execute, and the file will remain open!

See my gist for a self contained example: https://gist.github.com/ludamad/5579602

Avoiding problems

So what's the takeaway ? If you're going to fork & exec in a multi-threaded program, you cannot do any operations that require file-descriptors to be closed predictably. This is true even if only one of the threads does the process spawning!