Open in app

Sign in

Write

Sign in

Factom Core Dev Quickstart

Who
14 min readNov 23, 2020

--

After more than two years of core development on the Factom Protocol, I will be moving on. One of the goals I set for myself when I started my journey into Factom was to make it easier for people to follow along the same path. I wrote a significant amount of documentation for a variety of aspects but much of that was done with only partial knowledge of the beast.

This post is aimed at people who are interested in, or are hired for, core dev for the Factom Protocol. It is not intended as marketing material or to highlight the benefits of using Factom but as a no-nonsense introduction to the codebase.

Prerequisites assumed for this article:

  • Knowledge of golang
  • Experience working with factomd

If you aren’t familiar with how factomd works, you might want to read “Factom Quickstart for Developers who Don’t Know or Don’t Care what a Blockchain is” first to get started in the ecosystem.

Community

The Factom Protocol has a public forum and a public discord, which is where you’ll find most ANO members. The forum is where most community management (grants, governance, etc) and in-depth discussions happen, the discord is for everything else, including the TestNet management. The “Core Developers” forum contains a lot of useful resources.

There is a separate Factom Core Developer discord which is specifically for Factom developers. This is the place with the most resources for core devs and has channels dedicated to various aspects, such as the #code-review channel. All core devs should join this discord.

At the time of writing, there is also the separate Core Committee which has a private channel in the regular Factom discord. It deals with the MainNet, releases, and sort of a technical vision of the protocol. This is something you have to apply for separately, though that’s not necessary for regular core dev work.

The Code

Repos

Most of the code is located in the “FactomProject” GitHub, which is being managed by a few established members of the community. Historically, the repo was owned solely by Inc. up until recently, so not all important projects are in the FactomProject domain. The main ones are:

For core dev, you’ll almost exclusively be working with “factomd” and “factom”.

factomd

I previously created an overview of factomd, which still holds true, but doesn’t associate the concepts with the code. To briefly summarize: the heart of factomd is the engine, which starts a control panel and several FNodes, each of which has a State, a database, an API, and a P2P network. In a production environment, only one FNode is started, multiple FNodes are only used for simulations.

Internally, the FNode communicates through a bizarre structure of channels along which internal messages are sent. Figuring out where the messages are coming from and what exactly they do will likely take up a lot of your time.

Another important concept is that of the “process list” (PL for short). The PL is where the current block is being built and is frequently referenced to get the most up-to-date information. These are temporary and only exist for the block being built and a couple of the preceding ones.

The ultimate goal of factomd is to route new messages into the process list, and then collect all the necessary signatures from the authority nodes to sign off on a new block.

Project Structure

Taking a look at an upcoming version (6.12), I’m going to go through each folder in order of importance and describe roughly what it does.

Core:

You’re gonna spend most of your time looking at code in these folders on a day to day basis.

  • /common/interface: This is the central location that holds all interfaces widely used throughout factomd. You’ll end up having to modify these whenever you want to add new functions used by other packages. For more information, see the Interfaces section.
    Note: Despite the name, there are some structs mixed in with the interfaces
  • /engine: The heart of factomd. The engine handles starting up the node, from parsing command line parameters to initiating the FNodes, and the internal P2P network. Important files here are:
    - factomParams.go: parsing command line parameters
    - NetStart.go: initializing the FNodes/States with the config and starting all processes
    - NetworkProcessorNet.go: Routes incoming application messages from the network to various queues in State (and vice versa).
    - simControl.go: A monstrous 1,400 line switch that processes keyboard input on the factomd console for a variety of debug functions.
    - timer.go: Easy to overlook but this is what triggers end-of-minutes for nodes
  • /state: The heart of the core code that contains the brunt of the logic. Each FNode has exactly one state. Functionality is more or less just scattered around here, with most files contributing to the State struct in some way. Some important files:
    - authority.go: verify authority signatures
    - createEOM.go: send out internal EOMs
    - dbStateCatchup.go: if a node falls too far behind (or is syncing), “Catchup” will retrieve DBStates from peers
    - dbStateManager.go: The DBStateList contains downloaded or created DBStates and is responsible for applying the contents therein to State (“ProcessBlocks()”), as well as saving and signing them
    - entrysync.go: Downloads all missing entries from the network (usually referred to as “2nd pass”)
    - factoidstate.go: Holds the balances of all addresses
    - grants.go: Factom Grants are hardcoded here. Each grant round, the forum generates a template that core devs integrate and release in a new version. Every ANO must update in time as this is technically a hardfork.
    - HoldingList.go: the “holding list” contain messages that can’t yet be processed, like those that require Acks from Leaders before they go into the process list
    - identity.go: the State counterpart to the identity system.
    - loadDatabase.go: load DBStates from the database and also contains the code to generate the genesis block
    - MMR.go: Retrieves missing messages in the ProcessList
    - processList.go: The process list that is used to build new blocks. Each PL has a “VM” for each Leader. Each VM is a collection of messages added for that block. When messages have “Process()” called, this is what they work with. The PL will then update the State with its own “Process()” function
    - replay.go: The Replay filter that prevents duplicate messages
    - saveAndRestore.go: This is what’s used to save the current State to disk and generate the FastBoot file. If new features are added to State that contain information not saved in the database, it needs to be added here.
    - state.go: Has the State struct definition and an overwhelming amount of functionality for one file. Of particular note are: LoadConfig(), Init(), UpdateState() (this is what processes ProcessLists), GetNetworkBootStrapKey()/BootStrapIdentity
    - stateConsensus.go: Has code to validate, execute, and process incoming messages, and most of the functionality of messages that isn’t contained in the messages themselves, making it one of the most important files in the project for the consensus algorithm
    - stateDisplay.go: This is the “DisplayState” copy that gets sent to the control panel
    - stateFER.go: Controls the EC price
    - validation.go: Routes messages between queues and also creates the EOM when the timer ticks. Also has the node shutdown functionality
  • /common/[admin,directory,entry,entryCredit]Block: The very important structs for the internal node representation of those blocks. These are the objects that are serialized and stored in the database. This is the place where all blocks have their serialization code to for their binary formats and calculate hashes from. They are used throughout the codebase.
    (Note: the factoid block is in /common/factoid)
  • /common/factoid: Contains the FBlock code as well as functionality related to transactions, such as addresses and RCD definitions. Transactions are validated (signature and input/output checks) here.
  • /common/messages: Functionality to decode application-level messages and to execute their various tasks. All messages have “MsgBase” as a base, which is in /common/messages/msgbase. The subfolder “electionMsgs” contains similar functionality for election messages. For more information, see the Messages section.

Useful:

You’ll probably use these frequently.

  • /wsapi: The API (:8088) that can be used to interact with factomd. Only V2 is used these days. This is an amazing reference point you can use as an index for where many features are located. Want to find out where entries are stored? Just look at the code for the “entry” and you get the State’s entry lookup method as well as the database entry lookup.
    Also interesting is “wsapi.go” which contains middleware for the server.
  • /common/primitives: A dumping ground for basic shared functionality, which you’ll end up using a lot. Things like the marshaling Buffer, converting between human-readable keys and binary, the ubiquitous Hash definition, and Timestamp. Worth taking a look around in.
  • /util: Apart from miscellaneous functionality, this contains the default config file, parsing code, and the struct it parses into. If you need to add config file entries, this is the place to go.
  • /common/globals: This is mostly a globally accessible struct that holds the result of command-line parameter parsing (globals.Params). You’ll need this if you add more parameters.
  • /: The root folder contains a few interesting ReadMes. For actual core dev work, the files of note are:
    - Docker/Dockerfile.alpine: the docker files that are used to create the docker images that ANO pull from dockerhub (everyone uses alpine)
    - VERSION: contains the node’s version and should be updated with every release. There are no hard rules on numbering but the convention is major numbers for big hard forks, medium numbers for grants and backward compatible releases, minor numbers for small changes.
  • /.circleci: This is the configuration for Circle, which runs integration tests on pull requests, specifically build, go fmt, go vet, and go test.
  • /common/constants: An accumulation of constants in a central location. Important ones are: a list of valid application-level messages, ANO payout amounts and frequency, default network definitions, default chain definitions, and version number for the FastBoot file.

Relevant:

They are important for functionality but you’ll only have to modify them occasionally for changing those features specifically

  • /activations: Very small folder that only contains the functionality to activate specific features at specific heights for different networks. You will likely need this for feature-based hard forks.
  • /anchor: Contains the code to parse the anchor entries JSON format and verify the signature against the config setting. Submission of anchors is not part of factomd.
  • /common/identity: Functionality to decode the Factom Identity Chains and related functionality, like the identity manager. The identity manager is fed entries by the State and parses them.
  • /common/identityEntries: This deals with decoding the entries that ANOs submit for individual entities.
  • /controlPanel: Contains all of the code related to the node control panel (:8090). The control panel is a go web server running a custom html/js front-end with the “Foundation” css framework. It works by being passed a copy of State (“DisplayState”) every second, which is then polled via Ajax.
    The templates are compiled into a go binary via “staticfiles”, which you’ll need to do to make changes.
  • /database: Internally, factomd can use a variety of different databases, which are wrapped by the “database overlay” (/database/databaseOverlay). The overlay provides high-level functions for use throughout factomd. Data is stored as key-value pairs in “buckets” which are essentially prefixes for the keys.
    By default, factomd will use LevelDB to store the blockchain data and a BoltDB for a replay filter. You can specify the blockchain data db type with the “-db” command line parameter. It is very useful when developing to use “-db=Map” to use a temporary memory-only database for a clean chain.
  • /p2p: My rewrite of the P2P package, frequently dubbed P2P2. It’s a standalone package (also available separately) responsible for building the gossip network that connects nodes to each other. It can be treated as a black box where messages go in and come out. This package is extensively documented in the README.
    (Note: depending on when you read this, this folder may hold the outdated, old p2p package. At the time of writing, 6.12 is in the process of being rolled out to MainNet, which will contain P2P2)
  • /elections and /electionsCore: The code responsible for handling elections. Even after two years of working on factomd, this code is really hard to understand. It’s a mess of messages piggybacked onto other messages in multiple levels. The only person who really knows this code is Steven M, who no longer works on the Factom Protocol.

Other:

They do something but you probably won’t ever have to really look at the code unless something goes wrong

  • /events: The factomd side of the live-feed-api. It’s hooked into a few spots in factomd and (not enabled by default) will keep trying to connect via tcp to :8040 (default) and broadcast events via protobuf (see the “eventmessages” subfolder).
    It’s intended that you run the live-feed-api to interpret the messages but I wrote a debug utility that you can use to just print them out.
  • /modules: One of the reorganization attempts introduced the concept of “modules” which is supposed to be independent packages that can be included in factomd. At the moment, only the rewritten ChainHeadFix is in there.
  • /modules/chainheadfix: This is integrated into factomd (the -checkheads parameter, enabled by default) and its job is to parse all DBlocks to ensure that each “chain head” in the database points to the latest eblock of said chain.
  • /receipts: The code to generate and verify entry receipts, which is essentially just pulling all the relevant hashes to prove existence. The subfolder contains a standalone command-line app to calculate this without needing to run factomd.
  • /log: One of the custom logging features. This one will print messages to specific files that can be selectively enabled or disabled. For more information, see my post on logging in factomd.
  • /testHelper: Contains various support functions for unit tests, including the unit test node simulator.
  • /simTest: A collection of unit tests that use the built-in node simulator to run tests requiring multiple nodes.
  • /longTest: A couple of unit tests that take a long time to run.
  • /netTest: A collection of unit tests that only run against an already existing dev network.
  • /peerTest: A collection of unit tests that use the network. Tests are split into filename_A and filename_B, where both need to be started simultaneously in order for the test to work.

Useless:

They’re still around in the repo for historic reasons but serve little purpose nowadays

  • /Utilities: A collection of random utilities, most of them outdated, that should really be in their own repo. I don’t know if they’re still useful in any way.
  • /scripts: A file dump of random scripts. There’s no documentation for what any of these do and most are probably outdated. Try not to add anything to this folder.
  • /support/dev: An outdated ELK-Stack docker-compose setup that no one uses anymore. Inside the “factom” subfolder, there are three config files that are useful as crib sheets on how to configure running your own local network with multiple nodes.

State of the Code

I have mentioned this several times in the past but the Factomd code is very messy spaghetti code. When going in, you should definitely expect a lot of areas that just don’t make sense and could be easily improved. Unfortunately, this monster has grown to epic proportions over time and due to the entangled nature of everything, you need to have a very broad and deep understanding of the code to know if something can be touched “safely” or if it will have unforeseen consequences.

The code was initially created by Inc., who were unfamiliar with golang at the time, and didn’t follow any particular design. Most of the code is single-threaded and not thread-safe, particularly the consensus logic.

Interfaces

One of the things you’ll notice very quickly is that almost everything is an Interface (convention: starts with a capital I), even when it shouldn’t be. This was an intentional choice by the creator of Factomd to make the code more “object-oriented”, a decision he says he regrets.

Interfaces make sense in some cases, like IMsg, but do not for things like IHash (probably the most annoying interface), I*Blocks, and Timestamp (note: this isn’t called ITimestamp for some reason but it’s still an interface). Even State has a giant interface, IState, that mirrors many of the functions. This would be useful for unit tests, allowing the creation of “test states”, but in many places, the code will extract the *State reference out of the interface under the assumption it will work to get access to the underlying features.

Marshalling

There is a lot of code throughout the project dedicated to marshaling (a.k.a. serializing) objects into binary. Instead of using a framework, this is done via manual boilerplate code, of which there are over a hundred.

I have tried looking into what it would take to write an automated marshaling framework, but this is made difficult by the inconsistent way data is serialized. For example, in Factoid Transactions, the “input”, “output”, and “ec output” counter variables are all stored in the header instead of immediately preceding the list of items. This and other inconsistencies makes writing a generic unmarshaler fairly complex.

Factomd’s BinaryMarshallable interface is a custom one that also adds “UnmarshalBinaryData”. The purpose of this function is to only read what is necessary of the input and return what’s left. This is used in a few places where multiple messages are packed into one transmission but mostly not used.

Messages (IMsg)

The IMsg type is ubiquitous throughout the code and they are used for a variety of purposes, both for network transmissions and internal events. Figuring out what they do can be very confusing, as each message can perform multiple tasks, and there are messages with similar names to each other.

If you look at the IMsg interface, there are four important functions that define what messages do:

  • Validate: Defines if a message should be routed. Has three possible results: valid (1), can’t process yet (0), invalid (-1). Invalid messages are rejected outright, but ones that can’t be processed yet go into “Holding” and are continuously re-assessed until they become either valid or invalid.
  • LeaderExecute: The functionality that’s performed if the node is a Leader (Federated) and the message is meant for the VM assigned to that node for that minute. For most messages, this is just a wrapper for “FollowerExecute” (don’t do anything special) or “state.LeaderExecute”. The latter is a generic function which adds the message to the process list and sends out a signed Ack.
  • FollowerExecute: The functionality that’s performed by all nodes, including Leaders to whom the message is not assigned that minute. Most are just a wrapper for one of the state functions to handle functionality. The generic behavior (state.FollowerExecuteMsg) is to add it to Holding until both the message and its Ack have arrived. When both are present, the message is added to the process list at the right spot.
  • Process: The functionality of what happens when a message is inside a PL and the PL is being executed. This is the point where messages actually do work (adding entries, chains, transactions, etc). Messages that do not go into the PL will just have a call to panic() instead of implementing this.

Note: There are also election messages which, in addition to the above, also have election specific “Validate” and “Process” functions.

I have attempted to document what all the different messages do: https://forum.factomprotocol.org/threads/explanation-of-messages-imsg.1724/

The Future

After several years of development, the current version of Factomd is relatively stable with no known issues and no stalls for months. However, the community is in a transitional phase.

One of the proposed directions is Factom 2.0, a blockchain concept completely different from the current Factom Protocol. This would be a fantastic starting point for experienced developers, who could enter the project from the get-go and ensure that the project is designed to be maintainable.

If Factom 2.0 does not happen, it might be advantageous to rewrite the codebase. I have previously pitched an approach on how to do that. The downside is that you’d still need to be very familiar with the inner workings of Factomd to know what needs to be implemented.

Making major changes to Factomd is theoretically possible but there are very few truly independent parts to just replace.

Resources

Factom
Factom Protocol
Blockchain Technology
Blockchain

--

--

Who
Who

Written by Who

7 Followers
3 Following

Developer

No responses yet

Help

Status

About

Careers

Press

Blog

Privacy

Terms

Text to speech

Teams