Support for s3 object storage #541
Comments
|
Hello all 👋 All our integrations rely on the Hetzner Cloud public API, which is available with a certain level of stability. Since the features you are requesting are not in the public API, we cannot implement them. Therefore, for the time being, we do not plan to support:
Note that only a subset of the Amazon S3 features are currently supported. We will leave this ticket open to increase its visibility. If you have questions, reach out to us using the Support Center. |
|
@jooola A S3 compatible tool is a separate binary, not a library that you can use as part of your app. |
|
@hakman You are correct, I'll do a bit of research and try to come up with a few examples. |
|
@jooola Thanks. I was in a rush previously. Just for some background, someone reached out to us to ask about why he is getting an error when using Hetzner object storage with kOps. So there may be some incompatibilities with various clients.
|
Is this plan a short or long-term thing? I understand this limitation in the short term for reasons of reliability/development speed, but being able to work on this via the API will be important for widespread adoption. |
Could you please share a link to where this was reported? This would help us to take a look and check if it's a configuration issue or a genuine bug. Thanks! |
Creating a bucket programmatically is already doable using existing tools or libraries, the only missing part is to generate credentials using the public API. I agree that this is a limitation at the moment. Once we have made some necessary architectural changes, we should be able to work towards making this possible. However, we cannot give you a timeline at this point. |
|
Here are a few libraries examples that can be used to interact with our object storage. Note that only a subset of the Amazon S3 features are currently supported. Using Detailspackage main
import (
"context"
"log"
"strings"
"github.com/aws/aws-sdk-go-v2/aws"
"github.com/aws/aws-sdk-go-v2/credentials"
"github.com/aws/aws-sdk-go-v2/service/s3"
)
func main() {
ctx := context.Background()
// Create a new s3 client
client := s3.New(s3.Options{
AppID: "my-application/0.0.1",
Region: "fsn1",
BaseEndpoint: aws.String("https://fsn1.your-objectstorage.com"),
Credentials: credentials.StaticCredentialsProvider{Value: aws.Credentials{
AccessKeyID: "YOUR-ACCESS-KEY",
SecretAccessKey: "YOUR-SECRET-KEY",
}},
})
// Create a new bucket
_, err := client.CreateBucket(ctx, &s3.CreateBucketInput{
Bucket: aws.String("my-bucket-2bc3a910"),
})
if err != nil {
log.Fatal(err)
}
log.Printf("created bucket: my-bucket-2bc3a910")
// Upload a small object
body := strings.NewReader(strings.TrimSpace(`
# My object
This markdown document is the content of my object.
`))
_, err = client.PutObject(ctx, &s3.PutObjectInput{
Bucket: aws.String("my-bucket-2bc3a910"),
Key: aws.String("my-object-e9be8f5c"),
ContentType: aws.String("text/markdown"),
Body: body,
})
if err != nil {
log.Fatal(err)
}
log.Println("created object")
}Using Detailspackage main
import (
"context"
"log"
"strings"
"github.com/minio/minio-go/v7"
"github.com/minio/minio-go/v7/pkg/credentials"
)
func main() {
ctx := context.Background()
// Create a new s3 client
client, err := minio.New("fsn1.your-objectstorage.com", &minio.Options{
Secure: true,
Creds: credentials.NewStaticV4(
"YOUR-ACCESS-KEY",
"YOUR-SECRET-KEY",
"",
),
})
if err != nil {
log.Fatal(err)
}
// Create a new bucket
err = client.MakeBucket(ctx, "my-bucket-2bc3a910", minio.MakeBucketOptions{Region: "fsn1"})
if err != nil {
log.Fatal(err)
}
log.Printf("created bucket: my-bucket-2bc3a910")
// Upload a small object
body := strings.NewReader(strings.TrimSpace(`
# My object
This markdown document is the content of my object.
`))
_, err = client.PutObject(ctx,
"my-bucket-2bc3a910",
"my-object",
body,
body.Size(),
minio.PutObjectOptions{},
)
if err != nil {
log.Fatal(err)
}
log.Println("created object")
}Using Detailsimport logging
from boto3 import client
# Create a new s3 client
s3 = client(
"s3",
region_name="fsn1",
endpoint_url="https://fsn1.your-objectstorage.com",
aws_access_key_id="YOUR-ACCESS-KEY",
aws_secret_access_key="YOUR-SECRET-KEY",
)
# Create a new bucket
s3.create_bucket(Bucket="my-bucket-2bcxs3a0")
logging.info("created bucket: my-bucket-2bcxs3a0")
# Upload a small object
s3.put_object(
Bucket="my-bucket-2bcxs3a0",
Key="my-object",
Body="""
# My object
This markdown document is the content of my object.
""",
)
logging.info("created object")Let me know if you find a supported feature that is not working with one of the above SDKs. EDIT: I misclicked the auto close button 😬 |
@jooola Thank you for the examples and the list of supported features. I see that @LenzGr This was reported in the kubernetes/kops slack channel. I think Hetzner recommends using CAPI, but kOps does a pretty good job also. |
|
The examples were published in the official documentation: https://docs.hetzner.com/storage/object-storage/getting-started/using-libraries |
|
Thanks @jooola. Any plans to support |
I am not able to answer this question, you will have to ask the relevant team directly using the Support Center. |
TL;DR
When are we likely to get support for s3 object storage in the API/SDK/Terraform? With it scheduled to move out of beta into GA this week, I'd like to start using it in my applications.
Expected behavior
S3 support
The text was updated successfully, but these errors were encountered: