Demand Driven Architecture or REST or Linked Data?

I recently listened to David Nolen‘s talk from QCon London conference from back in July called Demand Driven Architecture. Before continuing, you should have a listen.


I really like a lot of things Mr. Nolen has done and really enjoy most of his talks and posts. I was less enthused with this one. I think my main hang up was his mis-representation of REST and resources. I get the feeling he equates resources with data stores. If you watched the video and then skimmed that Wikipedia page, you will quickly see that the notion of “joining” two resources is nonsensical. I think Mr. Nolen is really referring to that “pragmatic” definition that means POX + HTTP methods, which really would correlate well to data stores.

Real REST drives application state, so you would not need to join to anything else if using REST. His criticism of performance also misses the mark, for if you are using REST then you should also be carefully planning out and leveraging a caching strategy. REST isn’t appropriate for every application, but not for the reasons Mr. Nolen so casually dismisses it. Think about it this way: if REST is not suitable for performance or mobile devices, then you must also agree that all websites (not apps, just sites like fail on mobile devices. That’s just absurd.

I can’t imagine anyone is surprised when Mr. Nolen mentions SQL. He’s just mentioned joins, and what else would a developer associate with that term? If you have hung around the web long enough, you may have heard of SPARQL, which is a query language for linked data. Linked Data and SPARQL never seem to have caught on, but they address at least part of the problem Mr. Nolen presents. A big part of Linked Data’s failing is lack of tooling and painful implementation (mostly related to RDF). Perhaps new tools like BrightstarDB will help turn things around.

Interestingly, Mr. Nolen’s solution, SQL, is strong at ad hoc querying but not so great at scalability. If the web, linked data, etc. were already addressed by SQL, then those technologies would not exist. I really don’t get the need for mis-representation and indirection here. (On a related note, Fabian Pascal recently posted on the “Conflation & Logical-Physical Confusion” surrounding mis-representing SQL as the relational model, which ties in well to this post.)

The only thing presented here is an alternative to REST where the client specifies dependencies for the server to fulfill. This is a flip of the REST style where the server drives application state and provides contracts via media types to a client-driven approach. This is a perfectly valid approach, and interestingly one where Linked Data could excel. Tools such as Datomic, GraphQL + Relay, and Falcor certainly look interesting and appear to work well for very large projects.

I have no doubt that any of these techniques, done well, provides excellent results. Tooling will likely determine the winner, for better or worse.

Web API and Dynamic Data Access

In .NET Rocks episode 855, Jeff Fritz commented on ASP.NET Web API being somewhat confusing in terms of its intended use. I don’t tend to agree, but I thought I would address one point he made in particular: that Web API is perhaps just another form of repository.

Web API is much more than a repository. And yes, it is indeed a protocol mapping layer. As Uncle Bob once noted, a web or api front end is just a mapping layer and is not really your application.

In many cases, however, one could argue that a web-api-as-repository is a fairly solid use case. OData is a great example. However, I was thinking of yet another argument I’ve heard for dynamic languages: when you are just going from web to database and back, you are not really working with types.

In that spirit, I set out to write a simple Web API using SQL and JSON with no explicit class definitions. You can see the results in this gist:

I used Dapper to simplify the data access, though I just as well could have used Massive, PetaPoco, or Simple.Data. Mostly I wanted to use SQL, so I went with Dapper.

I also model bind to a JObject, which I immediately cast to dynamic. I use an anonymous object to supply the values for the parameters in the SQL statements, casting the fields from the dynamic object to satisfy Dapper.

All in all, I kinda like this. Everything is tiny, and I can work directly with SQL, which doesn’t bother me one bit. I have a single class to manage my data access and API translation, but the ultimate goal of each method is still small: retrieve data and present it over HTTP. That violates SRP, but I don’t mind in this case. The code above is not very testable, but with an API like this I’d be more inclined to do top level testing anyway. It’s just not deep enough to require a lot of very specific, low-level testing, IMHO.

Also, note again that this is just retrieving data and pushing it up through an API. This is not rocket science. An F# type provider over SQL would give a good enough sanity check. Why bother generating a bunch of types?

Which brings up another point for another post: what would I do if I needed to add some logic to process or transform the data I retrieved?

As a future exercise, I want to see what it would take to cap this with the Web API OData extensions. That could be fun.

LINQ to SQL and Entity Framework as Internal Object Databases

I’ve now used and/or tried LINQ to SQL, Entity Framework, and Fluent NHibernate and can now say I understand the differences as expressed in the ADO.NET Entity Framework Vote of No Confidence. Yet I still appreciate what the former two products from Microsoft offer. Well, I like LINQ to SQL anyway. After spending four hours today trying to create a simple example with Entity Framework and getting nowhere with many-to-many mappings despite several blogs’ assistance, I finally gave up. I think the problem with LINQ to SQL and Entity Framework is not in their usefulness but in the approach Microsoft has tried to take in marketing them as object-relational mapping technologies.

An object-relational mapping technology generally takes an object and maps it to a database, not the other way around. At least, I think that’s how it started. The abundance of MVC frameworks using the Active Record pattern seems to have changed that recently with generators creating models from database tables, though many of these actually create the tables in the database from the object definition, as well. Nevertheless, I’d disagree with Microsoft that LINQ to SQL and Entity Framework qualify as ORM technologies, though they do perform that role, as well.

LINQ to SQL and Entity Framework provide a language-integrated object database against which to create applications. This is huge! Let that sink in a bit. Now, I find nothing wrong with that approach. In fact, it’s quite nice! The trouble is that developers think, “Wow, I’ve got all these great objects ready to use!” Not so fast. You have entities that represent rows in tables, not business objects. Yes, LINQ to SQL and Entity Framework provide means of modifying those classes to mimic more class-like behavior (and that can indeed be a great benefit do easing domain model or active record development) but really should not be used for anything other than database records.

This greatly simplifies any data mapping you have to do between your domain objects and your database, and you can write everything in your OO language of choice. If you want something more automatic, you might try an object-to-object mapper (e.g. NBear–though I haven’t tried it myself and can’t imagine that object-to-object mapping would be that difficult).

As a final analysis, in case you care, I really like LINQ to SQL’s defaults. It’s super simple to get started and use, though it’s only for SQL Server. Entity Framework… I am just not a fan. If I can stay away, I will. Maybe someone will show me how to configure it so that it works for me, but so far it’s a FAIL. Also, keep tabs on DbLinq. It’s an open source tool that attempts to mimic LINQ to SQL for SQL Server and a number of other database technologies and should work on Mono. Of course, NHibernate is great for those who would rather connect to a real database, and I found Fluent NHibernate to be a great tool. I love its fluent interface for mapping to the database and its AutoMapping functionality. However, making everything virtual annoyed me and really makes me question how so many can prefer it for persistence ignorance when it so obviously requires that detail. (I get it’s a small sacrifice, but I wouldn’t code that way normally, so I am quite reminded that I’m connecting to a database through NHibernate.)

SQL Server Iterations with the Tally Table

SQL Server Central recently posted an article by Jeff Moden on using a Tally Table to iterate through database rows rather than a WHILE loop or CURSOR. The article defines tally tables and explains both how to create them and use them. Jeff even shows performance comparisons with a WHILE loop.

For those unfamiliar with tally tables, a tally table is generally used to parse data stored in a comma-delimited format but can also be used for other iterative actions. The tally table iterations also use set math and thus keep the CPU usage and row counts low consistently.

The next time you find yourself needing to iterate through table rows, try using a tally table.