S3 Use tcpdump httprequest find something wrong

[simpleshuai]

hresp, err := c.Do(hreq)
hreq:

Full Request URI with double hosts，how can I fix this？

[alimoeeny]

@wangmingshuai can you explain a bit more?

[simpleshuai]

@alimoeeny hi , thank you for focus
when i use s3 as the storage of my docker registry (v2), in the config file of registry,i use "us-gov-west-1" as the region,the config file as:
/data/config/config.yml

Before i start the registry，i use “tcpdump -i eno16777736 tcp port 80 -w newtcpdump” the capture some data，then i use “docker run -it --net=host -v /data/config:/etc/ceph registry:2 /etc/ceph/config.yml”,since i don't have valid accesskey and secretkey of aws s3, registry log print 403 forbidden.

But,when using wireshark to analysis the tcpdump ,i found the http request should be：
GET /test4/?delimiter=&marker=&max-keys=1&prefix= HTTP/1.1\r\n
But,it is:
GET http://s3-fips-us-gov-west-1.amazonaws.com/test4/?delimiter=&marker=&max-keys=1&prefix= HTTP/1.1\r\n
the Full request URI should be：
[Full request URI: http://s3-fips-us-gov-west-1.amazonaws.com/test4/?delimiter=&marker=&max-keys=1&prefix=]
But, it is:
[Full request URI: http://s3-fips-us-gov-west-1.amazonaws.comhttp://s3-fips-us-gov-west-1.amazonaws.com/test4/?delimiter=&marker=&max-keys=1&prefix=]

then,i want to find something, in s3/s3.go line 1154:
just before hresp, err := c.Do(hreq),i add code "fmt.Printf("-------before Do req=%+v\n",hreq)",then the result is：

---

actually,i want to use ceph-radosgw as the storage of registry v2,but registry abandon the way of "ceph-s3 " ,so i use the way of "s3",then i add a region in aws/regions.go ,and register it in aws/aws.go,when i start my registry ,the log show 400 bad request,just because the wrong thing we see in the tcpdump file;also ceph do not support "x-amz-copy-source-range",any idea?

[simpleshuai]

in s3/s3.go line 1108 :
hreq := http.Request{
URL: u,
Method: req.method,
ProtoMajor: 1,
ProtoMinor: 1,
Close: true,
Header: req.headers,
Form: req.params,
}

...
return &hreq,nill

i use the code below instead the code above , it worked!

hreq, _ := http.NewRequest(req.method, u.String(), nil)
hreq.Header = req.headers
hreq.Form = req.params

....

return hreq,nil

can you find something? @alimoeeny

[alimoeeny]

So, @wangmingshuai I am not sure if I understand the problem.
Is it the url? It seems to me that this url is malformed: http://s3-fips-us-gov-west-1.amazonaws.comhttp://s3-fips-us-gov-west-1.amazonaws.com/test4/?delimiter=&marker=&max-keys=1&prefix=
Right?
So about your solution, line 1108 of s3/s3.go is not like you say, it is https://github.com/AdRoll/goamz/blob/master/s3/s3.go#L1108

But in either case, if you want to send a pull request that solves the problem, specially if you can provide a test for it as well, I'd be very happy to accept it.

[simpleshuai]

@alimoeeny hi
The problem just is the malformed url that result in 400 bad request.
You can run the code below to get the problem:
tcpdump -i lo tcp port 4444 -w tcpdump
then in goamz/s3:
go test -bench TestPutBucket -v

now ,you use wireshark to analysis the dump file "tcpdump", you can find some bad requests:

[simpleshuai]

@alimoeeny
my problem is same as aws/aws-sdk-go#344