Macid is the HAppS storage mechanism that allows you to use whatever data structure you want + to hold your permanent data, without worrying about getting it into and out of tabular form + fit for storage in a traditional rdbms. + +
Before delving into how it works let's learn a bit about it from an operational perspective. + Assuming you have the tutorial installed and running locally: + +
The data you entered is stored on your filesystem in a directory under your running executable, called _local. + +
~/happs-tutorial>ls _local/happs-tutorial_state/
+
current-0000000000 events-0000000000 events-0000000001 events-0000000002
+
+
You can grep for it too. + +
+thartman@thartman-laptop:~/happs-tutorial>grep -ra testuser _local
+
_local/happs-tutorial_state/events-0000000000:ß\$6¡·¢:1525374391 696985193?AppStateSetBased.AddUsertestuser e1
+ ... (and lots more lines of binary data)
+
+
Hm... let's see, can we be sneaky and grep for the password? + +
Try it, you can't. Because the password is stored as an md5 hash, out of respect for the privacy of your users. + See the newUserPage function in ControllerPostActions if you're curious. + +
Keeping your macid data safe + + addfile ./templates/maciddatasafety.st hunk ./templates/maciddatasafety.st 1 +
All web startups have something in common. If you lose your user data, you are hosed. + +$!
So when I was first learning about HAppS, and contemplating doing a startup with it, one of my first questions + was, how do I keep this from happening? !$ + +
If you are using php, ruby on rails, or one of the other popular web frameworks, your user data is probably + in a mysql database, or if you are well funded maybe in Oracle. If you have outsourced your server hosting, + maybe you are even lucky enough to have a database administrator that takes backups for you on a regular basis. + That should help you sleep at night -- assuming that you can really trust that your dba is doing their job. + +
As we learned in the previous lesson, if you are using HAppS with macid, your data is + right there on your filesystem, by default in the directory called _local. + +
~/happs-tutorial>ls _local/happs-tutorial_state/
+
current-0000000000 events-0000000000 events-0000000001 events-0000000002
+
+
If there is money on the line, you are going to want to be careful with this directory. + +
If you have valuable data, and are migrating macid (lesson in progress) to a new data schema, + you are going to want to be extra cautious. + +
But for now, since you don't have any valuable data, the following procedure is probably enough + to remind yourself to be cautious while learning about HAppS in the tutorial sandbox. + +
Q: Do you have to shut down the HAppS server every time you migrate data to a new schema? + +
A: As far as I can tell, yes, you do, at least for the HAppS 0.9.2.1 release.
+
There is code under development in HAppS head that supposedly allows schema migration without
+ shutting down HAppS, by using multiple HAppS instances and storing your data in a
+ distributed way. But to keep things straightforward, in particular the zero-hassle
+ HAppS install, I'm not going to cover anything on this tutorial that hasn't been
+ released on hackage.
+
If your web startup requires zero downtime during data migrations,
+ HAppS probably isn't for you, at least not yet.
+ Then again, schema migrations using a traditional rdbms are
+ no picnic either.
+
+
Q: Is macid safe? Could I wake up one day with corrupted data under _local and no way to recover from it? +
A: Let's be realistic. + Compared to, say, mysql, HAppS hasn't been stress-tested much in critical high-volume web sites. + Or if it has, no one has shared their experiences. + (This is something I hope to change by making it easier to get started with HAppS.) + So whatever the HAppS developers say about reliability, personally I wouldn't be surprised if I encountered + some kind of data corruption problem as an early adopter. +
That said, the linux filesystem has been pretty well tested over the years. + And even though I am a linux fanboy I have to admit, you're probably + pretty safe keeping your data on the windows filesystem too. + Startup guru paul graham + famously created the web app + that would eventually become yahoo stores by just storing all application state in files. + + +
Taking a closer look at what is under _local... +
+thartman@thartman-laptop:~/happs-tutorial/_local/happs-tutorial_state>ls -lth
+
total 12K
+
-rw-r--r-- 1 thartman thartman 0 Oct 1 13:55 events-0000000003
+
-rw-r--r-- 1 thartman thartman 0 Oct 1 11:55 events-0000000002
+
-rw-r--r-- 1 thartman thartman 792 Oct 1 11:04 events-0000000001
+
-rw-r--r-- 1 thartman thartman 491 Oct 1 11:00 events-0000000000
+
-rw-r--r-- 1 thartman thartman 25 Oct 1 10:59 current-0000000000
+
thartman@thartman-laptop:~/happs-tutorial/_local/happs-tutorial_state>
+
+
+
My understanding is that macid serialization works by writing state change event data + one file at a time, and that by "replaying" the information here + in the order specified by the file names, you can recreate the state up to the time of the last write. + This is similar to the database transaction log + used by many rdbms systems. +
So, if I woke up one morning with my HAppS application in a corrupt, non-startable state and + my inbox full of angry customer email, probably what I would do is move files, one at a time, + out of the serialization directories, last-file created first, and keep trying to restart HAppS. + +
Q: What if my hard drive dies and I can't get my data back? +
A: Like with any other data storage system, if there's valuable data, you need to be making backups. + In the case of HAppS data stored under _local, I would probably be rsyncing + the _local directory to a remote server, or maybe multiple remote servers for extra safety. + For now I am not worried + about securing data, but when that day comes I'm pretty confident I'll be ok. + + + + + + addfile ./templates/maciddummydata.st hunk ./templates/maciddummydata.st 1 +
Assumng you are running this tutorial locally and everything worked, you now have a job board with no jobs -- +unless of course you took the time to create some some test data manually. + +
+When quickly hacking together a data-driven website, populating your app with dummy data is a chore you +want to avoid wasting time on, so let's see how to do that. + +
Firstly, you need to know how to back up your HAppS State data. hunk ./templates/menubar.st 5 -
And now, on to macid. hunk ./templates/toc.st 9 - , ("/tutorial/stringtemplate-basics","stringtemplate basics") + , ("/tutorial/stringtemplate-basics","stringtemplate basics") + , ("/tutorial/introductiontomacid","introduction-to-macid") + , ("/tutorial/macid-data-safety","macid data safety and backups") + , ("/tutorial/macid-dummy-data","macid dummy data") hunk ./todo 29 + + call it the happs zero hassle install (seo)