dev-tools:s3_object_storage

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
dev-tools:s3_object_storage [2025/09/25 19:27] – [Example Using mlflow] fabriciodev-tools:s3_object_storage [2025/10/15 16:37] (current) – [RKNS (Zarr V2) from Minio ZIP] fabricio
Line 243: Line 243:
 zarr.create_array(store=store, shape=(2,), dtype="float64") zarr.create_array(store=store, shape=(2,), dtype="float64")
  
 +</code>
 +
 +===== Example Using mlflow =====
 +This assumes that you have already set up a MLFLow server with a Minio artifact store.
 +In addition, the client has to also provide the credentials to the Minio store, as it directly sends it to s3, not via the mlflow.
 +You can do so by setting a few environment variables.
 +Note that the code below assumes you have a local file called 'ca.crt.cer', and that you have the credentials in an env file that is loaded via python-dotenv.
 +
 +<code python>
 +
 +print("\nLoading environment variables from .env file...")
 +load_dotenv()
 +
 +# --- Configuration: Bridge your .env to what MLflow/boto3 expects ---
 +os.environ["MLFLOW_TRACKING_URI"] = "https://mlflow.basel.lan"
 +os.environ["MLFLOW_S3_ENDPOINT_URL"] = "https://s3.strg1.lan"
 +os.environ["AWS_ACCESS_KEY_ID"] = os.getenv("STORAGE_ACCESS_KEY")
 +os.environ["AWS_SECRET_ACCESS_KEY"] = os.getenv("STORAGE_SECRET_KEY")
 +
 +ca_cert_path = "ca.crt.cer"
 +if not os.path.exists(ca_cert_path):
 +    raise FileNotFoundError(f"Certificate file not found at: {ca_cert_path}")
 +os.environ["AWS_CA_BUNDLE"] = ca_cert_path
 +# uncomment this if you want to ignore an insecure TLS/unsigned certificate
 +#os.environ["MLFLOW_TRACKING_INSECURE_TLS"] = "true"
 +os.environ["MLFLOW_TRACKING_CLIENT_CERT_PATH"] = ca_cert_path
 </code> </code>
  
Line 287: Line 313:
  
 </code> </code>
 +
 +
  
  • dev-tools/s3_object_storage.1758828452.txt.gz
  • Last modified: 2025/09/25 19:27
  • by fabricio