TLDR; I’ve created an open source project called Grimoire that serves two purposes: provide a digital companion for tabletop RPGs; and provide an exemplar for software engineering techniques that I have come to adopt over the past two decades of my career. Check it out at the github repository
A New Pet Project
Whenever someone asks how do I learn new technologies. I tell them that I take the time to implement a pet project using that technology or approach. Knowing the project inside out allows me to focus on learning the mechanics of implementing it with the new tools. Grimoire is a new pet project for me. And there are a lot of Software Engineering ( and even Software Business) concerns I want to touch on while building and discussing this application.
I don’t know where this experiment will go. Hopefully, the insights I share as I develop and discuss the project here and on social media will be useful for anyone who comes across it.
At the core, most business systems deal with Parties and how they interact with each other. There are other things that we model within a business system. But pretty much it comes down to Parties and Transactions:
A Customer makes a Purchase of a Product from a Retailer
A Supplier sends a Shipment of a Parcel to a Consignee at a Location
A Student creates an Enrollment in a Class taught by a Professor
A Doctor makes a Diagnosis on a Patient
A Customer makes a Payment to a Provider
A Candidate submits an Application for a Job at an Employer
Notice that each transaction has two parties participating. Also notice that in most cases, the parties have a different name. Giving entities within your system meaningful names is important. It’s one of the key concepts of Domain Driven Design: the Ubiquitous Language. The down and dirty details of the Ubiquitous Language is that your technical team and your business team should speak the same language and that language should be reflected in the code. This puts the onus on developers to learn the business speak (despite how funny Dilbert makes business speak sound). What it boils down to is that when you capture a concept from your requirements gathering, use the same word the domain experts use. There is no need for translation when you do this.
Also noteworthy that the Transactions are Nouns not verbs. Traditionally, when first being introduced to object design, we learn that Nouns are Classes and Verbs are Functions. We specifically made the transactions in our models nouns. A simplistic way of describing the transactions would have been :
A Customer purchases a Product from a Retailer
A Supplier ships a Parcel to a Consignee at a Location
A Student enrolls in a Class taught by a Professor
A Doctor diagnoses a Patient
A Customer pays a Provider
A Candidate applies for a Job at an Employer
Rather than representing the transactions as separate entities we are implying the transactions are simple functions on the entities involved in them. We’re losing some of the resolution of our model. An even more naive model follows:
A Party purchases a Product from a Party
A Party ships a Parcel to a Party at a Location
A Person enrolls in a Class taught by a Person
A Person diagnoses a Person
A Party pays a Party
A Person applies for a Job at an Organization
In the end, there is always a person or organization involved in transactions. But these parties have specific roles within the transaction. What’s more is that the same party might play different roles within different transactions. For instance, in a two ended retail system (think Amazon), it’s possible for me to be a customer in one purchase and a retailer in another. By separating the Roles from the Parties playing them, I’m able to distinguish between the two and I won’t overload my Party object with responsibilities. I know that the responsibility of the Customer is to make a Purchase.
So far we’ve covered three of the four archetypes in Color Modeling, the Party/Place/Thing (Parcel is a thing, location is a Location), the Role, and the Transaction. Actually I lied. We covered all four in a blink and you missed it moment. The Product in the Purchase transaction is not a Thing…not in our archetypal definition. The Product is actually a Description. If I purchase Assassin’s Creed for the Xbox from Amazon, yes I receive a physical thing. But that individual item is no different to the system than the hundreds of other copies of Assassin’s Creed for the Xbox Amazon might have in any of their dozens of warehouses. They are for all practical purposes fungible. That Product is an example of the Description Archetype. When the system cares about the identity of a specific item, like a VIN for a vehicle or a Serial Number for a Console, it becomes a Thing. The Parcel is a thing because it has a Tracking Number and is not fungible with other parcels being shipped.
Incipire has always been about startups. In fact the name of the company is the Latin word meaning to start. The idea to create a company focused on helping startups launch came from my experience with the Lean Startup Circle, a group that I stumbled upon purely through happenstance. I’d been learning about Lean Software Development through training from David Anderson (author of Kanban) and was searching for a discussion group around Lean Software Development when the google search turned up LSC. But finding the group, I found that their discussions around evidence based startups and treating the startup process as a series of experiments to discover your target market and create a solution that addresses their needs made perfect sense to me.
The Lean Startup model turns the typical concept of a startup on its ear. In the past startups were dependent on a business plan that were basically full of assumptions about the potential market, the reception to the product being developed, and the value that the startup could capture. You create a business plan, seek funding, and if successful use that funding to bring your startup to market in a way that supports your projections.
In the Startup Owner’s Manual, Steve Blank provides the example of WebVan, a dot-com darling whose goal was to revolutionize grocery shopping. Instead of going to the store, walking through aisles and waiting in line to pay for your groceries, shoppers could browse the site on the Internet and have their groceries delivered to them. Sounds like a great idea right? In fact there are companies (like Peapod) in existence today that follow that exact model. The problem with WebVan is that they vastly over-estimated their market and spent far too much anticipating a high demand. When actual demand didn’t meet their expectations, there was no way to adjust and WebVan filed for bankruptcy shortly after its IPO.
The problem with the traditional startup model is that it takes away the startup’s biggest advantage, being small and thus agile and makes it compete on its weakest point, money. A good investment round brings a startup maybe $25 Million dollars. Meanwhile for the companies whose market the the startup is trying to disrupt, that number might be a rounding error in a quarterly budget. The Lean Startup model focuses on minimizing waste (i.e. expenditure of time and money), validating assumptions, adjusting direction (or pivoting) to get back on target with what evidence says is accurate. In other words the Lean Startup Method emphasizes the strengths of a startup while minimizing the impact of its weaknesses.
In the ensuing years since I was first exposed to the concept, I’ve worked with a of companies launching new ventures either as new businesses or new products within an existing business to create what is called in Lean Startup terms a Minimum Viable Product. This could be anything from a simple prototype of an application that can be demonstrated to potential investors to a low-tech version of a service that would ideally be automated in the long run but can be sold as a manual service to clients. Getting an MVP launched allows you to get quick feedback on your market and its needs.
With that in mind, I’m taking what we’ve learned over these years and using that experience to launch a product which is essentially a tool that provides guidance to help startups launch their product in 13 weeks. I call this product, the Incipire Method: 13 Weeks to Launch. And what better way to demonstrate the method than to launch it over the next 13 weeks. Day one started Monday, today is day two. Over the next 13 weeks, I will document the creation of a new offering on the Incipire blog, as well as within a book dedicated to explaining the method. The software will be created as an open source product, but also sold as hosted, subscription-based web SaaS offering with a matching mobile application.
Why would I share my “secret sauce” with you. I believe that it will be genuinely useful for technical entrepreneurs who want to start a business but are overwhelmed by all that is required to learn to do so successfully. Basically, you can run as lean as you want if you’re willing to do it all yourself. I’ll show you how it’s possible to do so. I believe that Incipire offers enough value to the entrepreneur who wants our assistance in executing the method that we will be able to acquire customers who want to do so.
In essence, we are going to demonstrate the effectiveness of the Incipire Method by using the Incipire Method over the next 90 days. See you on May 22nd!
There are two concepts that are essential to any business software: what is being done; and who is doing it (a third concept is who or what something is being done to, with, or for). Before we can talk about anything else, we need to talk about people and how they fit into our system. People are so important to a model, they receive their own archetype in Peter Coad’s Color Modeling approach (well they share it with Organizations, Places, and Things but as we’ll see, they are conceptually interchangeable, because they are part of who is doing or who/what something is being done to, with, or for).
So how do you define a person? A Person entity has a Name (for the purpose of the series all entities are assumed to have an Id unless I state otherwise), and really that’s it. That’s all we have defined on our Person entity. Actually, some optional fields would include birthdate, Social Security Number (for the US, if you’re modeling a non-American person, SSN doesn’t apply). But in reality, I don’t think SSN belongs on a generic person model. There are only a few cases where you want to capture the SSN (for example in Human Resource/HCM applications), but as mentioned even in those cases, it’s not universally applicable. So our person entity ends up looking rather simple:
“Wait that’s it?” you ask. What about an address, a phone number, email address, SSN, phone number? What about all the information I need to gather for my customers, employees, and contractors? That’s a good question, or three actually. First and foremost, this is a conceptual model. What that that means is we’re going to break down these entities to their most basic components. Some times it will be prudent to simplify the models for the sake of performance (better queries, etc), I will leave that decision to you, the implementer. It’s best to be aware these raw elements exist so that should the need arise, you can extract them from a simplified model.
Before we go on to discuss the information that we might collect for a Person. Let’s look at the related Organization entity which can be a Legal Organization (such as a Company, Church, or School) or an Informal Organization like a Family, a Team or a Division in a Company. Again, take what you need, we’re doing a full tour for awareness.
Again the organization is spartan, containing only a name. Why don’t we add again address and other contact information to it? For simple models, it probably makes sense. But even for something as simple as a contact manager application, we’ll want to store multiple addresses and other forms of contact for a given person or organization. In order to simplify our model, because sometimes our system doesn’t want to differentiate between a Person and an Organization, we’ll add a common ancestor to both: the Party.
So back to the additional information we may want about a Party like the Address. Why not store it on the Party itself? Well like I said, what if we want to store more than one address? Or more than two? What if different addresses associated with a Party serve different purposes? For example as a Customer, I might have a Billing Address and a Shipping Address. But what if I want to sometimes send a Shipment to work? How does all of that work together? I have no less than 10 addresses stored in my account with an online retailer. Do I use them all on a regular basis? No, but I’m glad that they’re available when I need them.
What I’m getting at is rather than making an address an attribute on the Person entity (or even on the appropriate role), the Address should be a separate entity in and of itself. Len Silverston, David Hay, and Peter Coad all agree. Silverston and Hay cover these models pretty early. Coad talks about it pretty late in his book.
This model may seem overkill for getting a number for a given person. Like I said, you might want to elide some of the relations and entities for your model, but having more detail for clarity’s sake will allow you to simplify as needed. This model is the tip of the iceberg for the final model for Addresses in Silverston’s book, which goes into breaking down a postal address into a hierarchy of Geographic Regions. For most purposes if your storage mechanism supports GIS data structures, it will be simpler to leverage that than modeling the individual political and geographic hierarchies.
This has been a lot to consume (and share) for our first session. Look for Part 2 coming soon.
A question that I’ve answered a few times over the years is some variant on “how do I get better at Data/Domain Modeling?” I put an answer giving the resources I used to get better at modeling here on Quora, and am repeating it here. Over the next few…weeks? Months? I will be diving into these resources and walking through implementing a system built using the techniques provided in the books as we discuss them. So without further ado, I present an approach to become better at Data Modeling.
A good way to learn good design is to study good design. To that end, I recommend Len Silverston’s Data Model Resource Book. It comes in 3 volumes.
Volume 1 Covers Universal Data Models. The entities and relationships that are universally applicable no matter what project you’re building. A lot of these models form the foundation of CRM, ERP, HR, Accounting, Shipping, and Inventory Management systems so understanding them will help you understand the needs of people using those systems.
Volume 2 Covers domain/industry specific models. It builds on the Universal Models and adds new entities and relationships to cover basic needs for some of the primary industries:
Understanding these two sets of models will help you get a sound foundation in data modeling especially with the surrounding discussion about the significance of these models. What’s more, having studied them, you will be able to talk more knowledgeably about the given industries.
Volume 3 is where things get interesting, it provides a list of data modeling patterns and how to apply them to design robust and flexible data models.
Next up is David Hay’s work that builds on Silverston’s. He has 3 books:
: Presents an approach to data modeling including rules of thumb for making your data diagrams more organized and readable. He also presents a number of business entities and relationships that explore concepts within business a little more deeply than Silverston’s. You’ll see diagrams for contracts, work orders, lab work, document management. They supplement and augment the Volume 1 Universal Models quite well.
Data Model Patterns: A Metadata Map: basically talks about modeling a system for building software. If that makes any since. It takes the Zachman Architecture Framework and builds a model to support the concepts found therein. Basically if you want to build a system to manage software delivery processes, this is the place you should start looking.
Enterprise Model Patterns I honestly can’t tell you what’s in this book because I haven’t read it yet. I read the prior two before he released this one. But it’s on my backlog and you would probably do well to at least give it a passing glance.
Finally, comes the book that taught me everything I know about Object Modeling (and from it Data Modeling):
Java Modeling in Color with UML. Don’t let the name deceive you. This book presents an entirely new way of looking at data modeling. It is object-first, but you should be able to translate to relational models after your experience with the prior books in this list. Trust me, once you understand the methodology in this book, you’ll become a master of data/object modeling. The basic premise is that there are 4 archetypes in any data model. And those archetypes relate to each other in a specific pattern. And if you follow the heuristics of this approach, you can use that pattern to identify missing elements of your model. Not only that you can use it for discovery when talking to business stakeholders about features and concepts in the system you’re designing.
This leads very well into the topic of Domain Driven Design. But I think I’ve given you enough to keep you occupied for a while.
Here’s an excellent series of posts discussing how Github works. Not the website but the employees developing the website and supporting infrastructure.
He makes very excellent points in the series it’s worth a read but let me boil it down here.
Github is a distributed team, intentionally so (it helps them feel the pain of their target consumers). A traditional office environment (and in your case a Pseudo-office environment) is not very conducive to how creators work (developers are creators, just like painters, sculptors and writers we just use a different medium). Could you imagine if Michelangelo was told “you have to be here from 9-5 Monday to Friday to do your job”?
Having set hours to work stifles inspiration. I’ve had ideas bloom while taking a shower or relaxing on the couch during the weekend. Good luck having that magic moment recreate itself on Monday morning.
The “close quarters improvement” to collaboration is a lie sold by companies that wanted to save money on office space. If there is a problem that someone is having, when they’re sitting right next to you the tendency is to interrupt you. Basically what they’re doing takes precedent over your tasks. Removing proximity allows you to address another person’s needs on your schedule. In many cases the issues are resolved by the time you respond.
I’d say give a strong attempt at working in a distributed fashion. If you’re using Scrum, perhaps come together during the daily stand ups, sprint review, and sprint planning sessions. Other than that, you don’t need instant communications with each other.
I had a quick back and forth with Clemens Vasters via twitter about his views concerning MQTT. I definitely understand his frustrations at the way the protocol is being marketed, but my view as a newcomer to the “IoT” environment was that MQTT is a perimeter protocol to make it easier to get a message into your system from a simple device. The only thing an MQTT endpoint should be doing in my eyes is handing off the message to an internal bus so that it can be further processed.
Clemens mentioned to me that is not how it’s being sold and we both agree that it does have its place as a perimeter protocol. This brings me to my big point. Your infrastructure should not adhere to an external protocol. Protocols are for interop, and as such are limited to what a committee of implementors can agree on. Internally, you should be using a much more efficient and targeted means of communication. Take for instance the OBD-II protocol. Automakers are required to support it by law, but internally, they use their own messaging layer that is much more efficient and of course supports more functionality.
Too often I have seen projects pick up the next big buzzword and adopt it without concern as to its best application. SOAP/WSDL is one example. When I hear people asking if they should implement their internal services using SOAP when internally they’re using a homogenous environment it makes me want to pull my hair out (of course this is not as frequent because SOAP isn’t en vogue anymore. Now everyone wants to be RESTful although some of the very things that the RESTafarians were railing against (bloated protocol) are making their way into RESTful frameworks. It’s as if they’re realizing that Metadata for automatic discovery of an API might be a good thing (go figure).
But I digress. I once worked on a project for a major player in the financial sector and their entire application used FIX internally to communicate between components. Gone were the concepts of Object-orientation, if you wanted a property of a particular message, you had to query it from the message structure. To make it simple, it was not the most efficient process (and this was for a realtime system for Forex trading where milliseconds in delay can be the difference in making a profit and eating a loss).
If someone is selling you a protocol as a solution to build your product around, you should ask yourself what’s in it for them. Better yet, tell them “I’ll get back to you on that.” If you want to support a protocol in order to simplify integration with other products make sure that the protocol stays where it belongs, on the outside of your system.
My daughter told me on her own out of the blue that she wants to learn how to program. As a life-long computer nerd, this development made me elated! I have been hoping that one of my progeny would express an interest in computing but I didn’t want to push them there. I wanted the interest to be self-motivated.
I remember quite well my first experience with programming. It involved typing a bunch of code in BASIC from the back of Family Computing Magazine and Compute! so that I could have a free game. When I first started I had no idea what it was I was typing. Just that I would get a game when I was done.
It was this visceral link that fueled my passion for programming. Type something in get a reward, see the results. I wanted to give a similar experience to my daughter while also letting me to join on the journey of wonder with her. After careful thought and consideration, I decided that the Arduino would provide a perfect environment to learn not just programming but also electronics.
While I’m waiting for the Arduino starter kit that I ordered to be delivered (due to arrive on Monday), I decided to look through the book I selected as a tutorial. I noticed that it spent a lot of time on the basics of programming (which I don’t need) while also giving good coverage to electronics. But what I really wanted was a “do this, see results” tutorial that was less pedagogical and more like mimicry.
Think of how we learn as babies/toddlers. We learn to walk by copying the big people around us who always stand upright. We learn to talk by reproducing the sounds these people keep making around us. No one tells us the mechanics of walking, we learn through trial and error. When we start talking, we don’t understand grammar, we just keep seeing the big lady call herself mama and that’s an easy sound to reproduce so we start saying it. And repeating it when she gets excited that we said it.
Again it’s very visceral. I do something and get a result. I try to take a step, encouraged by my mother and manage to take a few before losing balance and falling on my rump. I type in code on my computer and get a working program. I don’t necessarily understand algorithms or processors or memory allocation, functions or any of the deeper why’s or how’s to programming. I just know that following someone’s instructions gets me a free game to play.
It’s the feedback of success for our guided effort that encourages us to continue learning more. I think that a young me having to sit through a course on computer architecture and the theory of computing before I could use a computer would have lost interest and I would have missed my calling. (Okay I lie, I read the DOS manual in my spare time when my family first got our computer).
I don’t want to burden my daughter with the theory of electronics and computing. I want to give her that guided effort/success feedback loop that I had as a child so that I can get her excited about her success and ready to keep learning.
When we start doing something, it’s important to get that positive result as soon as possible. Whether it’s learning to walk, program, or play an instrument, we all like to have some feedback as to whether we’re on the right track.