An honest answer is: I don't know for sure. +
To be honest, maybe not. hunk ./templates/macidstresstest.st 10 -
I have done some preliminary testing though. +
I have done some preliminary testing, and so far the results have not been good. + +
I am hoping that I am doing something wrong, and that there is a way of using macid effectively + for more than just toy applications. I also asked the HAppS googlegroup for help, and if there is a + solution for the problems I found I will definitely be sharing it in the tutorial, so stay tuned. + +
In what's written below, ControllerStressTests.hs + has the implementation of the tests. hunk ./templates/macidstresstest.st 22 - two users and two hundreds jobs in a single transaction -- one macid state file got written. + two users and two hundreds jobs in a single transaction -- the set up of users got transformed + in a single update. + +
What happens if we push harder? + +
If you click on stresstest/200 the result is that 200 users + get inserted, each with 200 jobs. (But don't do that!) + Each insertion is its own event, so macid winds up having to do a lot of work. + Also, the number in the url controls the number of users inserted -- it's always 200 jobs per user + no matter what. + +
Result for me (notes taken during the experiment):
+
Okay, started 0:44.... inserting about 1 user every 5 seconds. chugging along... slowly...
+ I'm not seming multiple event files getting created, instead one very large event file is growing under _local.
+ Perhaps there's an event file for each time happs stop-starts?
+ Five minutes later, we've inserted 100 out of 200 users and my laptop is VERY hot.
+ There's only 3 files under happs-tutorial_state (we started from pristine, no users no jobs) but
+ one of the files is enormous. 15 minutes later the insertion is completed.
+
+
thartman@thartman-laptop:~/happs-tuorial/_local>ls -lth happs-tutorial_state/events-0000000000
+
-rw-r--r-- 1 thartman thartman 356M Oct 5 00:54 happs-tutorial_state/events-0000000000
+
+
Seems like a lot a lot of hard disk for what would amount to about 40,000 data records, if we were + using sql instead of macid. Does that mean that if I had a million users (monster.com) + I would need 356M * 5000 = 178G of hard disk space... and also that much ram? + I think we have terabyte disks now, but I haven't heard of terabyte ram -- maybe in the google cloud? + +
Even granted loads of ram there's another problem, maybe related. + +
If I shut down and restart happs, it's slow to start. hunk ./templates/macidstresstest.st 53 -
Let's now push harder. +
time wget http://localhost:5001 shows two minute page retrieval page retrival for the home page, + which doesn't even call macid. When I attempt to re-retrieve that or other pages, it continues to take + minutes -- so it's not a startup issue. Overall my laptop is very sluggish. HAppS seems to have stolen + all my ram. hunk ./templates/macidstresstest.st 58 -
If you click on stresstesttwohundie the result is that 200 users - get inserted, each with 200 jobs. Each insertion is its own event, so you wind up with over +
When I repeated the experiment with less ambitious stress tests, eg click /tutorial/strestest/10 + or even just a few (2-3) users at a time, inserts were also very slow, and I experienced the same slowness + starting happs and retrieving the homepage. + +
40,000 some-odd data records doesn't seem all that big to me for a web-app, and the real limit appears to be + even lower than that. + +
Granted, I thought too hard about my data structures, and pretty much used the first thing + that popped into my head just to "get coding". Could I be seeing better performance if I used smarter + data structures? + +
Are there other issues I need to think about? + +
Or is it back to databases if you want to build monster.com? + +
If I'm going to be using happs in real applications, which I would like to do, + getting happs-tutorial to be able to run these types of stress tests without feeling + an overwhelming urge to call + pkill ghc and free up my laptop ram is essential. + +$! + +you wind up with over hunk ./templates/macidstresstest.st 88 -
Okay, started 0:44.... inserting about 1 user every 5 seconds. chugging along... sloowly... - no files appear to be getting created under _local. Perhaps this only happens after the serverpart exits? - Five minutes later, we've inserted 100 out of 200 users and my laptop is VERY hot. - There's only 3 files under happs-tutorial_state (we started from pristine, no users no jobs) but - one of the files is enormous. hunk ./templates/macidstresstest.st 89 -
thartman@thartman-laptop:~/happs-tuorial/_local>ls -lth happs-tutorial_state/events-0000000000
-
-rw-r--r-- 1 thartman thartman 356M Oct 5 00:54 happs-tutorial_state/events-0000000000
+
+
hunk ./templates/macidstresstest.st 102
-
+!$