Jun 20, 2022
[dev] code highlight
Hi, I'm Naoki Komoto (河本 直起) working as Machine Learning Engineer in AnyMind.
In AnyMind, we’ve just started introducing machine learning functions to our products. So we are developing our MLOps environment, including the data pipeline from scratch.
In this article, I'd like to introduce our data pipeline using Cloud Composer (Airflow) including its current set up and future plans.
The dataset for the machine learning (ML) model training is originally stored in the product application’s RDB. When I joined the team, model training applicatio