January 5, 2019

2018 Technical Summary

Here is the summary of the 2018 year from the technical perspective.

Tool(s) of the year

Initially Enlightenment was the only one candidate, but in the end of the year I have discovered f.lux, and it immediately has been promoted into the "Tools of the year".

Language of the year

I was skeptical about the language in the beginning, but after working with it more, and getting some practical experience changed my mind, and indeed was the pleasant discovery of the year.

Conference of the year

Although being honest I have attended only 2: Joker in Saint Petersburg and HighLoad++ in Moscow/Skolkovo, but the later triggered more discoveries and thoughts.

Library of the year

Very nice library. We have been using Spotify Docker client for a while, but migrated to the Testcontainers.

Commitment of the year

Since February, 1st I have set the commitment to start working on my open source projects every day. Even 5-10 minutes, but everyday. So there should be the commit/activity every single day in the GitHub contributions track:

2018 Contributions Overview

There are equal number of commits and issues, as new ideas and features are coming at the speed of the actual implementation, and I use the GitHub Issues as the tasks tracker for the projects.

There are 4 main projects I have been working through the past year:

All of them came out of the necessity, i.e. they were built to help me (and the team) to work on the day to day development activities on the main project.

vmx

  • Description: Remote instances management tool over SSH written in Go
  • Lang: Go
  • Status: Ready to use

vmx activities summary

This tool is used in the project to do dev and prod deployments, plus various handy commands to run against specific set of servers.

Likely, I think, there could be other tools available for doing similar tasks. And even all our infrastructure is automated using Ansible, there is still a place for the vmx tool. At least currently we are running 3 dev EC2 instances, and about 21 prod EC2 instances, and vmx does the job well, and I am not looking for other tools.

Ultimate next step would be to introduce Kubernetes to project, considering we are running all the services as the Docker containers since the very beginning, but in the current stage Kubernetes would not bring any technical or business values to the project.

uuidfmt

  • Description:
  • Lang: Go
  • Status: Ready to use

uuidfmt activities summary

This tool was written to mainly convert between 2 different formats of UUID-s. One format is used in the MySQL, and another format is used in the DynamoDB (and these are the primary keys in the tables), so it was cumbersome to do the conversion manually. And the tool was born.

Also built the Snap of the tool, and released into the store:

Get it from the Snap Store

And some metrics of the Snap image usages and territories:

uuidfmt snap usages

uuidfmt snap territories

I am glad that other people also found the tool useful, not many, but still.

dynosql

  • Description:
  • Lang: Java
  • Status: In development

dynosql activities summary

The idea was to implement the SQL parser and to transform from SQL to the corresponding DynamoDB query/scan. There are no really good libraries for this in the Java ecosystem available. There is similar library in Python with its own defined like SQL language - DQL. And also RazorSQL provides support for running SQL queries against DynamoDB.

I have run multiple stages in the implementation, from trying to build the parser using ANTLR and the available ANTLR SQLite grammar. It did not work well. Was trying to use some of the available libraries for parsing SQL query. Also none of them really fit my need. So finally, as in the first stage the support is only limited to the SELECT statements, and they will be extended by the syntax similar to the DQL, I do parsing of the query myself.

The goal is to use this library as the dependency for the dynoman, but also could be used standalone, so this is why it was extracted into the separate project.

dynoman

  • Description:
  • Lang: Kotlin
  • Status: Ready to use

dynoman activities summary

Actually this project was the main focus of the year, and the time I spent mostly on.

Again was built for the necessity to use more convenient and features rich tool to work with DynamoDB rather than official web UI/console. Actually the UI is modelled after the web UI, but with some extended features I was missing.

Proved to work fine, as in 80% of the cases (the more I progress, the more cases it covers) I use this tool instead of the web UI.

Also it was the nice opportunity to practice Kotlin, and look into the modern Java desktop development (I had good experience in the past working with Swing in my student years). And really enjoy building desktop apps.

Plans for 2019

What I am sure I will continue to do is to work on my open source projects:

  • provide good solid documentation for the vmx
  • complete dynosql to be able parse DQL and produce the corresponding DynamoDB query
  • deliver the dynoman to the stage with the core features I am aiming for
  • provide the various distribution formats for the dynoman to be installed on Mac, Linux and Windows

From the learning new things perspective:

  • will continue to explore Kotlin more
  • get some good introduction to the Machine Learning, not necessary to become the expert, but getting good understanding of the tools and concepts, and getting some practical experience

Other things are more nice to do, like to explore Clickhouse and Tarantool (again from the perspective to use them in the project), but the above is already a lot, considering I do them in my free time, and have the full time job with lots of challenges.

Tags: practice