...
We constantly try to improve based on input from our developers and improve our Dashboard with the help of the feedback from our customers.
...
Our backend chapter believes in micro-services. We would like to scale up our system as freely as possible. The main flavor of our services in Node.Js and Typescriptis Typescript with Express. For our persistence storage, we chose MongoDB and we and use Elasticsearch to search through large volumes of data quickly. All services communicate with HTTP requests with our interpretation of REST practices. Our main data set has over 200 GB of data so get ready to experience some scale. 🚀 We use a variation of REST as the main means of communication between services, and pub/sub topics for more async functionality. Observability and ownership are very important concepts for us, we use Sentry, Grafana Dashboards, Pingdom, Elastic, and Argocd to monitor our system.
We always strive to take a modern and pragmatic approach to system design. We are working on introducing GraphQL API gateway to enrich frontend with some cool features and reduce the complexity in loading multiple resources from backend. Leveraging real time streaming to ElasticSearch, aggregate caches and future GraphQL API gateway we are always on a lookout for performance increase, we aim for loosely coupled solutions that scale. There are multiple active initiatives around increasing isolation between workflows and services.
Machine-Learning
...
We take the advantages of Tensorflow machine learning framework for creating AI models and the model trainings are performed on our own AI model training system which utilises dynamically provisioned Google Cloud VMs and Redis queues. This schema allows us to run concurrent AI model trainings without tackling scalability issues.
...