sometimes things don't work, it's not your fault and you cannot fix it. And sometimes this comes in close succession. So here is my 2nd annual rant for this year:
Chapter I
I'm working on a backup daemon, because i'd like to have a copy of important data outside of my home, just in case. As usual this takes some days until it works, i make local tests and fix bugs and try to move the software to a good state. Then tests with a connection over the internet; and finally this works too.
After that i thought of reasonable extensions to the daemon. 5 GB of webspace come to mind. They are mounted via WebDAV and can be accesses as any local file system. Nearly.
First i had to adjust the character set for encrypted file names until this seemed to be no longer the cause of transmission problems.
Then: symbolic links are not supported. Ok, it's mostly for files only anyway, so i accepted this.
Next is: i cannot set the modification date on files. I use this as a flag whether the file is fully synced. If mdate and file length match, then the file is not further scrutinized but deemed to be up to date. I modified the backup daemon to accept any mdate which is newer than the original file's mdate.
Next is: frequent errors, irregular with no pattern. Except: 9 of 10 errors are timeout errors. I work around to catch and retry in case of timeout and this seems to work. Running the daemon for some time updated more and more files to there final size. But some files failed over and over again.
Common pattern: long files. Approx. 15 files and they all were in range 10 to 50 MB. I examined the problem further: It seems that data written to a WebDAV file is not uploaded immediately, but only buffered internally. When the file is closed all data is uploaded. And, unbelievable, the close() system call has a timeout. It's generous: 1 minute. And then it fails with timeout error... and nothing is saved! So there's no real chance to upload long files this way.
I tried to enforce upload in chunks by calling fsync() after each chunk of data. What the WebDAV driver does here is: it uploads all the current file, not only the modified data, and now, after a long while, while it uploads the growing file again and again, it fails with timeout error when it reaches the critical file size. :-/
Here i gave up. WebDAV is not supported.
Chapter II
That was last week. This week i developed the idea, that i could make daily snapshots of the current backup and hard link the files to save space. After all, 'Time Machine' does it as well. And we are Unix, aren't we? After two days it worked, tested locally. What i got back from my friend was:
Client1: caught error: create hardlink "/Volumes/K/1/2013-07-15/,NSMHyr<1 {i>+_AbQ5Tc<89I3t0>Er''dGEb(xZNjJ{!#yiO4rmSE8_ILhE": Operation not supported (45)
(funny filenames, eh?)
Yes: create hard link not supported. He has a Mac like me and stores my backup on his NAS. The NAS is a Linux box and surely supports hard links. So it's the file transfer protocol: AFP. After 12 or 13 years of being unixoid Apple did not manage to implement hard links into their Apple File Protocol. This sucks! But how does 'Time Machine' do this? After all, they not only create hard linked files but also directories (according to internet sources) which is generally deprecated!
'Time Machine' creates a SparseImage filesystem file on the NAS and mounts this locally. Wow! That's what i call a workaround. So my friend can just do the same and never forget to mount this volume, or i discard the daily snapshot feature. But i don't want to drop it, so i rewrote the daemon to ignore the failure of creating a hard-linked snapshot of the current backup and just resume work. The risk is, that in case i need the backup it may just have been crippled to the state on my Mac before i can restore it... :-/
Chapter III
In my work i write apps for Android. Today i wrote a screen to input a bunch of dates and times. I didn't want the DatePicker and the TimePicker to popup over and over again, so the user should be able to enter date and time in text edit fields. There are specializations for these and similar cases, and i flagged the text fields as 'date' and 'time'. 'date' works. 'time' does not allow me to enter a colon. Actually, not any character besides '0' to '9'. Has Google ever tested this? There is no workaround or solution for this problem, so i used a nasty workaround and require the user to enter a period instead of a colon... :-/
And here i felt it's time for my second annual rant this year.
... Kio !
now i feel better. ;-)
No comments:
Post a Comment