Mabel API Is Mostly Built Out Now

I've been hard at work since the last blog post building out the Mabel api. I finished that task yesterday morning. Since then, I've been burning in all the new endpoints and fixing the various bugs that particular process uncovered. 

At this point, the following tasks remain:

• Code the validators for the endpoints that accept post bodies.
• Write all the unit tests for the new/updated code in the application.

After those two tasks are complete, it'll be time to move back to the client and build out all the code it'll need to make requests to Mabel and handles her responses.

Gotta Set Up My Debugger Again

 I can't believe I spent a couple of years not being able to debug code I was running on remote servers. I'm glad this is something I've learned how to do. Those years of debugging via logging were excruciating.

Anyhow, my project in PHPStorm got terminally . . . hosed . . . a few weeks ago and I had to reset it. Something went horribly wrong with the file indexing.

Today I needed to use the debugger again for the first time since that and that meant I had to configure it again.

Luckily, that was easy to do as I found this still germane walk-through from four years ago.

That Feeling When A Bug Makes You Happy

It's not very often that you come across a bug in your code where, after tracking down the source of the problem, you're pleased that the bug exists.

According to the documentation on redis.io, the hgetall command returns all the fields and values of a hash at the given key. However, the command result is a list where each field is followed by the corresponding value. 

I assumed that calling this method in predis would return a result formatted the same way and I baked this assumption into my code.

When I hit an endpoint that executed this code for the first time the api returned an error response and I could see an exception in my log file. Looking more closely at the stack trace, I could see that Mabel was attempting to access numeric array indexes that didn't exist.

I threw in some logging and saw that, contrary to my earlier assumption, predis was automatically translating the result of the hgetall command into an associative array.

This is great news. I didn't mind working with the result format described in the redis.io documentation, but I knew that if I were dealing with larger hashes I might start to run into problems. Working with associative arrays is much easier.

So yeah, sometimes a bug in your code can be a good thing.

A Use Case For Middleware

With the exception of /anonymous/news, every valid request to Mabel has to go through the "check if the authorization header is in the request, get the header value, is the value an empty string, yada yada yada" rigmarole.

Initially, I put this authorization functionality into the beginning of the action() methods of the controllers I was building out. Eventually, I noticed commonalities and moved that code up to the base class that all controllers inherit.

It became clear though that I was going to end up in a situation where controllers of the same authorization case would contain identical code at the start of their action methods. However, I would not be able to move that code up to the base class without implementing some really 'clever' code. And as I always say, "there's good clever and there's bad clever" and this would clearly have been a case of the bad flavor.

As it so happens, this is the scenario where middleware really shines. Middleware is simply the idea that I can tell Slim (or other similar micro-frameworks) that I want code to execute before or after the action in a controller. In some frameworks you explicitly state "I want this code to execute before" or  "I want this code to execute after". In Slim 4 you sort of just wrap your middleware around the action and handle the 'before or after' part in the middleware itself.

Ultimately, I was able to move all of the repetitive code out of the controllers entirely, leaving behind just the code that pertains specifically to that controller's action.

So yeah. Yay middleware.

 

Refactor Everything. Then Refactor Everything Again.

This post has been very difficult to write. In some sense, the process of writing it mirrored what the next step of building out Mabel was really like. I knew that the end result of the post was going to be a description of the final User model that I developed and how it worked. Similarly, I knew that the end result of "the next step in building out Mabel" would result in a concise, unified and rational framework for managing persisted user data. What I didn't know in either case was how to get from start to finish.

With respect to developing the User model, what really helped was being able to generate the following set of requirements that the above mentioned framework must implement:

1. If I load user data via Redis\User I must do so with the value of the authorization header.
2. If I load user data via MySQL\User, I want to save the data via Redis\User so it can be used later as the fast source.
3. However, in some cases I need to perform validation on the data loaded via MySQL\User before I save it via Redis\User.
4. In some cases I wanted to load data via MySQL\User with the User Name field. 
5. In some cases I want to load data via MySQL\User with the User Id field.
6. When I load data via MySQL\User, I want to save the User Id via Redis\Authorization as cases exist where there would be no other way to link the an authorized header value to the actual user account.
7. When I save data via MySQL\User and am creating a new record, I need to confirm that the User Name, Screen Name and Email Address do not exist for any record currently in the database.
8. When I save data via MySQL\User and I am updating a new record, I need to confirm that the User Name, Screen Name and Email Address do not exist for any record currently in the database EXCEPT for the record of the current user.
9. When I save data via MySQL\User, I want to save the data via Redis\User so it can be used later as the fast source.
10. When I save data via MySQL\User, I want to save the User Id via Redis\Authorization as cases exist where there would be no other way to link an authorized header value to the actual user account.

Once I fully understood what the requirements were, coming up with the implementation was pretty easy. Generally speaking, the code is structured as follows:

• There are three user models
• Models\User
• Models\Redis\User
• Models\Database\User
• Models\User requires the other two as constructor parameters.
• All three models have identical public getters/setters for user data.
• Models\User has three methods that can be used to load user data:
• loadFromUserKey
• loadFromUserName
• loadFromUserId
• Models\User has two methods that can be used to save user data:
• saveToRedis
• saveToDatabase
• It's the controller's responsibility to manage which load or save methods to call and in which order. 
• It's the responsibility of Models\User to manage the transfer of data between itself and Redis\User or Database\User as appropriate.
• It's the responsibility of Database\User to manage the special validation that occurs when saving user data to the database.
• It's the responsibility of Database\User and Redis\User to manage the actual communication with their respective services.

It came together pretty neatly in the end and I feel pretty good about it. There's very little duplication of code. I expect it'll be relatively straightforward to unit test and it's not terribly confusing code to read through.

Definitely Going To Implement A Fast/Slow Persistence Model

One of the things that I want to implement for Mabel is fast/slow data persistence. 

It works as follows:

• When data needs to be loaded, the first thing that's checked is a 'fast' persistent store like Redis. If the desired data isn't located in the fast store a slow store, like a MySQL database, is then checked.
• When data is saved, the data is first saved to the slow store. Then the old data in the fast store is replaced with the newer.

Why would I want to implement this? Well, slow is . . . slow. In the case of a MySQL database, you really don't want every single SELECT statement in production to execute against the database, especially since a large number of those statements return identical result sets. Might as well cache those identical result sets somewhere and take the load off the system.

If I were going to do a full implementation of a fast/slow system I would also put the slow system updates behind some sort of a queuing tech like RabbitMQ or something bespoke built on top of Redis. That's really only necessary when you're very very concerned about your slow store being overwhelmed by the number of writes it's being asked to do or the update to the slow store needs to trigger other back-end processes that you don't want handled synchronously (like sending emails).

At this point in time, that's not necessary so I'm not going to build it.

But yah never know. Maybe some day . . .

How Exactly Is Authorization Going To Work?

After reorganizing Mabel to reflect the decision to move my dependency declarations out of docroot/index.php and to switch to a 'one endpoint one class' approach, it was time to dig back in and go to an even greater level of detail.

Authorization seemed like a good place to start but I also wanted to flesh out the data model for user information, so I began with POST /account/create.

The business logic for this controller should be something like:

1. Confirm that the request has the authorization header and throw an exception, log it and return a 400 Computer Says No response if it doesn't.
2. Get the value of the authorization header.
3. If the value is an empty string, generate a new value and save it to persistent storage in an unauthorized state.
4. If the value is not an empty string, attempt to load it from persistent storage.
5. If the value cannot be loaded from persistent storage, generate a new value and save it to persistent storage in an unauthorized state.
6. If the value exists in persistent storage in an authorized state return a 403 Computer Says Forbidden response. You should not be attempting to create a new user account if you're already authorized.
7. If the value exists in persistent storage in an unauthorized state we're golden. Continue on.
8. Get the post data from the body of the request and translate it into something the code can work with.
9. Validate the translated data for the existence of all required fields, that those fields don't contain invalid data and that they're neither too long or too short.
10. If the translated data failed validation, return a 406 Computer Says Not Acceptable response.
11. Confirm that no other already persisted user has the same User Name, Screen Name or Email Address included in the translated data.
12. If one or more of these three fields are already in use, return a 406 Computer Says Not Acceptable response.
13. Save the translated data to persistent storage, thus creating the new user account.
14. Update the authorization header value saved in persistent storage to an authorized state.
15. Return a 200 OK response.

So yeah, that's basically what's happening behind the scenes for that particular endpoint. The other endpoints are largely variations on the same theme.