One of the upsides to being quarantined is I don’t have to feel bad about spending a long weekend alone coding. It’s amazing to think what projects are being hatched as we speak! Somewhere in the world there’s a private discord channel where the future is being worked out right now.
On that note, I’m proud to announce that I blissfully whisked away my own weekend building a Codenames app to play with friends and family! You can see it in action here. The repo is here. There’s tons of alternatives freely available already, but I thought it would be a fun way to put a couple ideas I’ve been kicking around to the test. I would like to go over some of the design decisions and takeaways.
The Data
The database specification is a traditional attribute-style Datascript/Datahike schema and is fully shared between client and server. It includes a somewhat involved model consisting of the kinds of things you’d expect of a simple game: players, users, games, rounds, turns, and so on. Having a shared data model is perhaps the thing I enjoyed most about this exercise. I am finding it’s fruitful to think about what portion of the application should be shared in .cljc files. It helps reinforce a–yes–data-oriented mindset.
The Layout
The layout is a bit unique for this application, so I thought I would talk about it first. It is fully declaratively described as data using a library I started called swig. It’s a recursive layout engine that is all about “user-driven” UI. In other words, right now it’s basically lame WYSIWYG. Built on top of the excellent re-com, it aspires to be an extremely simple, stick-anything-in-anything kind of layout system. It’s very alpha, but so far it has been working out well.
The layout spec looks something like this:
(def game-layout
(swig/view {:swig/ident :swig/root-view
:swig.view/active-tab [:swig/ident tabs/pregame]}
(swig/view {:swig/ident tabs/app-root
:swig.view/active-tab [:swig/ident tabs/pregame]}
(swig/tab {:swig/ident tabs/users
:swig.tab/label {:swig/type :swig.type/cell
:swig.cell/element "Users"}})
(swig/tab {:swig/ident idents/chat
:swig.tab/label {:swig/type :swig.type/cell
:swig.cell/element "Chat"}
:swig.tab/ops [{:swig/type :swig.type/operation
:swig.operation/name :operation/fullscreen}
{:swig/type :swig.type/operation
:swig.operation/name :operation/divide-horizontal}]})
(swig/tab {:swig/ident tabs/leader-board
:swig.tab/label {:swig/type :swig.type/cell
:swig.cell/element "Leader Board"}
:swig.tab/ops [{:swig/type :swig.type/operation
:swig.operation/name :operation/fullscreen}
{:swig/type :swig.type/operation
:swig.operation/name :operation/divide-horizontal}]})
(swig/tab {:swig/ident tabs/game
:swig.tab/label {:swig/type :swig.type/cell
:swig.cell/element "Game"}
:swig.tab/ops [{:swig/type :swig.type/operation
:swig.operation/name :operation/fullscreen}
{:swig/type :swig.type/operation
:swig.operation/name :operation/divide-horizontal}]}
(swig/split {:swig/ident splits/game-split
:swig.split/orientation :vertical
:swig.split/split-percent 30
:swig.split/ops [{:swig/type :swig.type/operation
:swig.operation/name :operation/join}]}
(swig/view {:swig.view/active-tab [:swig/ident tabs/player-board]}
(swig/tab {:swig/ident tabs/player-board
:swig.tab/label {:swig/type :swig.type/cell
:swig.cell/element "Players"}
:swig.tab/ops [{:swig/type :swig.type/operation
:swig.operation/name :operation/divide-vertical}]}))
(swig/view {:swig.view/active-tab [:swig/ident tabs/game-board]}
(swig/tab {:swig/ident tabs/game-board
:swig.tab/label {:swig/type :swig.type/cell
:swig.cell/element "Board"}
:swig.tab/ops [{:swig/type :swig.type/operation
:swig.operation/name :operation/fullscreen}
{:swig/type :swig.type/operation
:swig.operation/name :operation/divide-vertical}]}))))
(swig/tab {:swig/ident tabs/pregame
:swig.tab/label {:swig/type :swig.type/cell
:swig.cell/element "Pregame"}}))))
The functions are just thin wrappers that return hiccup data. This spec is trivially compiled to a “normalized” set of Datascript facts with something like this:
(defn hiccup->facts
([hiccup]
(hiccup->facts -100000 hiccup))
([parent hiccup]
(let [id (volatile! -1)
conformed (s/conform ::node hiccup)] ;; ::node is a simple recursive spec for hiccup.
((fn run [parent idx hiccup]
(lazy-seq
(when (seq hiccup)
(match hiccup
[(:or :element :empty-element) props]
(let [id (or (-> props :args :db/id) (vreset! id (dec @id)))]
(conj (vec (mapcat (partial run id)
(range)
(match (:body props)
[:nodes nodes] nodes
[:nodes-list nodes] nodes)))
(cond-> (assoc (:args props)
:swig.ref/parent parent
:db/id id
:swig/index idx
:swig/type (:name props))
(not= parent -100000)
(assoc :swig.ref/parent parent))))
:cljs.spec.alpha/invalid
(s/explain ::node hiccup)))))
parent 0 conformed))))
This is ugly code, but it would be easy enough to make a better compiler. From here, you can imagine it’s simple to recursively project out a reagent view. This is done currently with a single mutually recursive multimethod definition that is Swig’s render function.
Some things to note about this general approach:
-
It’s handy to be able to navigate up, down, and around your graph starting from any starting point. This is essential for cleanly expressing significant UI transforms. UIs (the kinds I’m talking about) are path-independent by nature, so transforms are best described in a path independent way. To that end, Datalog is a beautiful way to marry functional programming with graph based programming. Ever felt the pain of trying to keep track of where elements are in your render tree? Maybe you’ve tried tracking their positions by path? Does not work. Or maybe you’ve been tempted to put entitry tracking info in the DOM? Yikes. In this system, you can easily give elements a proper, context-free identity that you can refer to in your event handlers and does not need to flow through the entire render tree (which tends to be brittle).
- It’s trivial to persist the high-level UI state to the back-end. Just register
a transaction listener and use the phenomenal
Sente to
send deltas to the server. For example:
(defn tx-listener [{:keys [db-after tx-data tx-meta]}] (chsk-send! [:codenames.comms/facts {:datoms tx-data :tx-meta tx-meta}]))And you’re done. This is a great place to also filter by attribute and attach any metadata you might care about. Note that these UI states are updated transactionally, making it (relatively) trivial to avoid an unfortunate client disconnect leading to an inconsistent UI state.
- Trivially customize the view to hardware/device. Tabs can be moved anywhere
with a transaction as simple as:
[{:swig/ident ::me :swig.ref/parent [:swig/ident ::my-new-parent]}]These transforms can be run client side or server side.
-
Swig (aspirationally speaking) handles the hard parts. Layout engines are tricky. Bootstrap is very powerful, but it’s insufficient for many applications, requiring developers to hand-role a layout system or hard-code it in a brittle view tree.
-
The layout is data. In reagent, the result of a view function is a mixture of code and data. Using Swig, the mixture is pushed to the leaves. The top levels of the layout are specified declaratively.
-
Integrations. There’s potentially a lot of 3rd-party integrations that would be possible using a layout framework like this.
- Plug in to Datomic for great leverage.
The Subs
With logic-based subscriptions, we can spend most of our time in a happy world thinking relationally about our data, then cleanly map the model to our views. For example, consider this query for player type (i.e. codemaster or guesser):
(def-sub ::player-type
[:find ?player-type .
:in $ ?game-id
:where
[?sid :session/user ?uid]
[?game-id :game/teams ?tid]
[?tid :codenames.team/players ?pid]
[?pid :codenames.player/user ?uid]
[?pid :codenames.player/type ?player-type]])
If this doesn’t make sense to you, I’d suggest hopping over to Learn Datalog Today and doing the exercises. You can learn the basics in an afternoon. In the meantime, think of Datalog queries as expressing a logic-driven traversal through a graph of data in a direction invariant way.
You might be thinking the model implied here is a little over-complicated, consisting of
users, games, players, teams, etc., but you can see that at least we can still express a
traversal through this graph cleanly, and in a maintainable way. It’s scary to
think of the nested get-ins we might reach to in a traditional re-frame app.
Consider a more complicated query: the end-state of a Codenames game. A Codenames game is over when a team guesses all of their cards (resulting in a win), or when a team incorrectly guesses the assassin card (resulting in a loss). We can express that condition nicely with a query:
(def-sub ::game-over
[:find [?team-id ?color]
:in $ ?game-id
:where
(or (and [?game-id :game/teams ?team-id]
[?id :codenames.character-card/role]
[?game-id :game/current-round ?round-id]
[?team-id :codenames.team/color ?color]
(or (and [?round-id :codenames.round/blue-cards-count 0]
[?team-id :codenames.team/color :blue])
(and [?round-id :codenames.round/red-cards-count 0]
[?team-id :codenames.team/color :red])))
(and [?game-id :game/teams ?team-id]
[?game-id :game/current-round ?round-id]
[?team-id :codenames.team/color ?color]
[?id :codenames.character-card/played? true]
[?id :codenames.character-card/role :assassin]
[?id :codenames.piece/round ?round-id]
[?round-id :codenames.round/current-team ?team-id]))])
Could be a bit cleaner, but not too bad. Admittedly, in a real app you might not use such a complicated query for performance reasons, but it’s neat that such a thing can be expressed so readily.
The Events
In re-posh, events are simply Datascript transactions. Datalog provides an ideal system for declaring entities and their relations. In my view it’s unfortunate actually that a lot of Clojure programs suffer dramatically from the exact problem that Datalog beautifully solves: path-dependence. We’re suffering every time we write global, deeply nested app states that we access with nested get-ins. Such code becomes brittle, path-dependent. Datascript suggests a way out of this mess on the front-end.
In some cases, the performance may appear a bit cringy, considering we’re creating a lot of transient “transaction data” that will need to be mapped to datoms. But bare in mind that you can feed datoms directly as tx-data, which are directly dumped into the database. However, it ends up that in many cases you want the added expressivity of transactions-as-data. In any case, the scalability of a reactive logic system on the font-end remains an open question.
Business Logic
Consider the logic to create a new game:
(def-event-ds ::new-game [db _]
(let [session (d/entity db [:swig/ident idents/session])
user (:session/user session)
teams [(assoc (utils/make-team "Blue Team"
:blue
[(utils/make-player (:db/id user) :codemaster)])
:db/id -2)
(assoc (utils/make-team "Red Team" :red [])
:db/id -3)]
first-team (-> teams shuffle first)
first-color (:codenames.team/color first-team)]
(into [{:game/finished? false
:game/current-round -4
:game/rounds -4
:game/teams teams
:game/id (utils/make-random-uuid)
:db/id -1}
{:swig/ident idents/session
:session/game -1}
{:codenames.round/number 1
:codenames.round/current-turn -5
:codenames.round/blue-cards-count (case first-color :blue 9 8)
:codenames.round/red-cards-count (case first-color :red 9 8)
:codenames.round/current-team (:db/id first-team)
:db/id -4}
{:db/id -5
:codenames.turn/team (:db/id first-team)
:codenames.turn/word ""
:codenames.turn/submitted? false}]
(utils/make-game-pieces -4 db/words db/board-dimensions first-color))))
Here we succinctly:
-
Create a game, player, teams, round, and game pieces.
-
Link our entities. Games have rounds and teams. Teams have players. Every player has a user. Rounds have turns. etc. Datascript, like Datomic, uses negative numbers to specify entity relations. I find this to be one of its most brilliant features.
Most likely you’d want to move most of this into constructor functions. But the main thing is I don’t think it would be great to stuff this data into some kind of nested map.
Finally, consider how trivial it would be to move this event logic to the back-end. That applies really across the entire system, even up to server side rendering.
Layout Transforms
It’s easy to express layout transforms with Swig by simply moving parent refs.
(def-event-ds ::enter-game [db [_ tab-id game-id]]
(let [tab (d/entity db tab-id)
parent (:swig.ref/parent tab)
parent-id (:db/id parent)]
(into [[:db.fn/retractAttribute tab-id :swig.ref/parent]
[:db/add parent-id :swig.view/active-tab [:swig/ident tabs/game]]
[:db/add [:swig/ident tabs/game] :swig.ref/parent parent-id]
[:db/add [:swig/ident idents/session] :session/game game-id]])))
This will make the “pregame” tab disappear and the “game” tab appear in its place. I’m not sure about the ultimate efficacy of this approach to transitioning UI layout, but it has been interesting to play with.
Versioning
As a slight aside, I think versioning is where the rubber meets the road here. What should the granularity of versioning be? Should every entity type be versioned independently? Perhaps, but I also think Datalog offers an interesting approach here. It takes the view that there basically shouldn’t be versions. There is just immutable attribute specs. You then write code to be attribute specific, not entity specific. It’s interesting approach that starts to make a lot more sense as you spend more time with Datalog. Datalog really is key to the whole idea.
The downside of this approach is it tends to encourage use of domain specific, fully qualified names where you might otherwise use generic names for generic functionality. This is something to watch out for when embracing fully generic container types.
The Views
Views in this system are the familiar ratoms that we know and love. However, Swig controls the dispatching to view methods. For example, here is the game-board tab:
(defmethod swig-view/dispatch tabs/game-board
[{tab-id :db/id}
(when-let [game-id @(re-posh/subscribe [::session-subs/game])]
(let [cards @(re-posh/subscribe [::game-subs/word-cards game-id])]
[v-box
:width "100%"
:children
[[board-info tab-id game-id cards]
[turn-handler game-id]
[scroller
:style {:flex "1 1 0%"}
:child
[board-grid game-id cards]]]])))
tabs/game-board is just a keyword that we associate with the tab in the layout
definition shown above. The Swig render will invoke this view method when
it encounters it while rendering. Thus we’ve decoupled our view from the
layout. Handlers can be registered at any level of the layout tree, so you can
still generate highly customized html.
The Ugly
This is all still a little Janky. It’s hacked on top of re-posh, which hacks posh into re-frame, which is an impressive hack of an incremental view maintenance system into Datascript. These are all fantastic libraries that have helped me understand the value of logic systems, but It would be cool to see a beautiful ground-up implementation of these ideas (probably built on top of Datascript). There has been some Art in this area, but nothing has fully matured or caught on. I recommend taking a look at them.
The other major problem is there’s no solution for routing yet. I’ve thought about different approaches here, but the best solution is likely to be application specific.
One interesting feature of this is that “snapshotting” a session is trivial. You could easily code up a system for session sharing in a way that’s easy to maintain as the system evolves. You could then imagine a page linking and sharing mechanism based around session snappshotting, similar to what you see in many SPA’s today.
Conclusions
After using Datalog in various projects for a couple months now, I find myself less swayed by other client-side query languages or frameworks, whether it be Falcor, GraphQL, Relay, EQL, the list goes on. I’m especially wary of coupling components to query–this doesn’t make sense to me. Finally, I’m more wary of excessive preoccupation with “shape”, as that creates path dependence.
One thing I think you especially don’t want to do is role your own half-baked Datascript in the form of some “normalized” model of the view. If you want a persistent, normalized model of your view–almost however simple it is–Datascript is the way to go. In fact, ultimately there’s no reason you can’t replace React with a Datascript transaction listener if you’re feeling adventurous. Move the transactor into its own WebWorker process and now you’re really talking.
Overall, I’m somewhat bullish on logic systems as an improvement on conditionals generally, but time will tell whether they grow in use (so far it has told us they won’t). In any case, there’s still a lot of unknowns about the viability of Datalog on the front-end. For small projects that aren’t highly demanding, I think it’s a powerful tool that might provide a lot of leverage to a small team. For demanding apps, a proper RETE implementation for Datalog might be necessary. There’s some interesting research on the topic here. Or maybe you’re smarter than me and you can figure out how to just use Clara Rules idiomatically to solve this.
Maybe we should hop into a private discord channel and get to work…