KREONET

Development of KREONET AI Research Platform

Objective

• The goal of this project is to provide an environment where researchers working on AI can directly utilize large-scale data—delivered through high-speed networks—for computing and experimentation. This enables seamless exeion of AI research using real-time data access.

Funding

KISTI Basic Research Program: Establishment, Operation, and Service of Science and Technology Research Network Infrastructure Based on Future Demands

Description

1. KREONET AI Research Platform

• Definition: The KREONET AI research platform is designed to support AI researchers and member institutions by providing a Science DMZ–based high-speed network for fast data transfers. During non-transfer periods, the platform allows idle resources to be utilized for AI computing tasks.
• Architecture: ① Science DMZ Network (for high-speed big data transfer) + ② AI Research Platform Interface (computing environment for researchers)

  - Science DMZ Network Configuration (Application Service): This component provides an end-to-end, lossless high-speed data transfer infrastructure using DTN (Data Transfer Node) servers and a Science DMZ network specifically optimized for research workloads.

  - AI Research Environment Platform (User Interface): A system is built on DTN hardware that automatically allocates computing and storage resources to researchers, offering a seamless AI computing experience.

2. Development Environment for the KREONET AI Research Platform

• An open-source–based AI platform is developed using the following technologies:

  - An AI research workspace is built using a containerized virtualization environment based on Kubernetes and Docker.

  - The container environment utilizes Ceph’s object storage interface for storage. This setup allows integration with external cloud systems and enables data transfer using the S3 interface, providing a cloud-compatible and extensible storage infrastructure.

  - A private image registry is configured using Harbor, enabling users to store and share container images in a secure and controlled environment, similar to Docker Hub.

  - The platform provides a distributed computing environment using Kubeflow’s TF-Operator and MPI Job, with access currently available through a CLI-based interface.

  * A user-friendly environment is needed, which requires the development of a web-based interface.

  - Keycloak is used to provide user authentication, and Harbor and Grafana are configured to support a centralized and unified login environment.

• KREONET AI Platform – AI KREONET is a system designed to provide computing environments, consisting of computing, storage, distributed computing, and monitoring components.