ResponseError: 401: Unauthorized after a long run

[ncw]

I've been trying to use it to backup a large (200GB, 40000 files) directory

./cloud-sync.py -d DEBUG -T 1 -f /tmp/log -u xxxxxxxxxxxxxxxx -k xxxxxxxx --authurl https://auth.storage.memset.com/v1.0 /home/xxxxxxxxxxxx/xxxxxx/ swift://xxxxxx

It works but after a while it produces this error

2011-10-17 16:28:52,986 cloud-sync   DEBUG    5345565 completed of 5349661 - 99%
2011-10-17 16:28:52,986 cloud-sync   DEBUG    5345565 completed of 5349661 - 99%
2011-10-17 16:28:53,259 cloud-sync   INFO     xxxx/xxxxxxxxxxxxxxxxxxxxxx/xxxxxxxxxxxxxxxxxxxxx.xxx completed
2011-10-17 16:28:53,259 cloud-sync   DEBUG    Returning Connection to the pool
2011-10-17 16:28:53,259 cloud-sync   DEBUG    Run 2497
2011-10-17 16:28:53,260 cloud-sync   DEBUG    Getting Connection
2011-10-17 16:28:53,260 cloud-sync   INFO     Saving cf://xxxxxx:xxxx/xxxxxxxxxxxxxxxxxxxxxx/xxxxxxxxxxxxxxxxx.xxx to /home/xxxxxxxxxxxx/xxxxxx/xxxx/xxxxxxxxxxxxxxxxxxxxxx/xxxxxxxxxxxxxxxxx.xxx
2011-10-17 16:28:54,500 cloud-sync   DEBUG    Returning Connection to the pool
Exception in thread Thread-1:
Traceback (most recent call last):
  File "/usr/lib/python2.7/threading.py", line 552, in __bootstrap_inner
    self.run()
  File "./cloud-sync.py", line 22, in run
    self.work()
  File "./cloud-sync.py", line 34, in work
    quote(task['file'],'/'))
  File "/home/ncw/Code/cloudfiles-sync/cloud_providers/swift.py", line 179, in put
    connection.get_container(container).create_object(remote).load_from_filename(local,callback=self.callback)
  File "/usr/lib/pymodules/python2.7/cloudfiles/connection.py", line 341, in get_container
    raise ResponseError(response.status, response.reason)
ResponseError: 401: Unauthorized

I think this is happening when the auth token expires after one hour. The python-cloudfiles module has code to get a new token when this happens but I suspect either your use of threading or ConnectionPool is breaking it. I'm using version 1.7.9.2 of the python-cloudfiles module.

[welbymcroberts]

Hi Nick

Thanks for pointing this out. I'm going to try and get a few moments to have a look over this sometime this week. The python-cloudfiles module should be trapping this error, however I should be able to trap this within the Swift cloudprovider as well.

Cheers
Welby