Neptune is a cloud-native data science lab that enables individual and teams of data scientists to run multiple experiments simultaneously, thus shortening the time required to achieve their results. It also gives you the ability to collaborate, share results and manage the model training process.
Building machine learning models requires heavy computing power. When an experiment is launched, the need for computing power skyrockets. By the end of the experiment, however, little power at all is needed. But to fully adress the needs of data science teams, the tool had to:
Neptune can accommodate computing-heavy data experiments while at the same time relieving users of the need to invest in proprietary hardware.
Neptune enables data scientists to run several experiments at once, and compare the results in order to cherry-pick the best performing models. To that end, it uses Kubernetes and virtual machines (VMs) that can be deployed and closed on demand. Thanks to its cloud-native design, training datasets can be shared among the VMs running the experiments.
experiments done in the cloud
users
hours of GPU virtual machines used
availability
Neptune runs on Kubernetes, ensuring that the process of establishing and closing the VMs is smooth. By using templates provided by Helm and handling its tasks in real time, Neptune reduces the time needed to run new machines and start an experiment. Users need not pay to run a machine while it is being established, but only for resources when they are being used.
Further, by leveraging the distributed MooseFS, Neptune ensures that all VMs share access to the training dataset being used, making additional storage for every machine unnecessary. Last but not least, Kubernetes makes Neptune infrastructure-agnostic, so it can be established in both a private or public cloud. Neptune can be run on a laptop, using cloud resources, or on a bare-metal infrastructure.
Building machine learning (ML) models requires heavy computing power. When a data scientist launches an experiment, the need for computing power skyrockets. By the end of the experiment, however, little power at all is needed.
It should come as no surprise, then, that many data scientists and companies building ML-based solutions move to cloud to buy on-the-go computing power and pay only for resources. The cost of on-premise infrastructure is just too high for many to cover.
Neptune runs on Kubernetes, ensuring that the process of establishing and closing the VM is smooth. By using the templates provided by Helm, Neptune reduces the time needed to run new machines and start the experiment, as everything is done in real time. Users need not pay to run a machine while it is being established, but only for resources when they are being used.
Neptune enables data scientists to run multiple experiments simultaneously, with every experiment launched on a separate machine with multiple GPUs, designed specially to run the machine learning process. By leveraging the distributed MooseFS, Neptune ensured that all the VMs share access to the training dataset, making additional storage for every machine unnecessary.
By using Kubernetes, the app is infrastructure-agnostic and may be established in both a private or public cloud. Neptune be run on a laptop, using cloud resources, or bare-metal infrastructure.
Neptune was developed for internal use by CodiLime to support its own data science team. As it proved supremely convenient, effective and scalable, the company decided to launch it as an independent product, and later spun the company off.
Designing cloud-native applications is the optimal approach to today’s business challenges. By embracing the advantages provided by cloud computing and modern distributed infrastructure, companies are able to scale up, manage and secure their processes.
If you’d like to discuss the details, contact us!
