irc.oftc.net #zumastor log beginning Tue Apr 1 00:00:01 PDT 2008 2008-04-01 00:16 12:15... Dana is 4 years and 15 minutes old now 2008-04-01 00:23 She was born on midnight on April 1st? 2008-04-01 01:14 rounding everything to the nearest year 2008-04-01 01:14 then calculating the difference since the age changed 2008-04-01 01:24 flips: the nearest *year*? 2008-04-01 01:26 "has been 4 years old for 15 minutes" 2008-04-01 01:26 precise enough? 2008-04-01 01:26 got it 2008-04-01 01:26 I just never realized that she was born on April 1st. 2008-04-01 01:26 you're going to believe that? 2008-04-01 01:27 ;) 2008-04-01 01:27 I'm kind of in a state of limited credulity until the end of the day. 2008-04-01 01:27 in this case its true 2008-04-01 01:27 That is either going to be a lot of fun for her or horribly annoying. 2008-04-01 01:27 she waited for an auspicious date and that was it 2008-04-01 01:28 I think the former 2008-04-01 01:28 cbsmit, this makefile stinks ;) 2008-04-01 01:28 flips: Ick... not again! 2008-04-01 01:29 more like, still 2008-04-01 01:29 ACTION silently prays this is an April 1st thing. 2008-04-01 01:29 Gotta move the unit test library over, I know that. 2008-04-01 01:29 it's serviceable, but it has those ddsnap binaries still in it even though ddsnap has moved to another directory 2008-04-01 01:29 bundh of things like that 2008-04-01 01:30 Wha? When was ddsnap moved to another directory? 2008-04-01 01:30 ddraid I mean 2008-04-01 01:30 always mix those two up 2008-04-01 01:30 Oh yeah, that stuff should be removed, along with the relevant source files. 2008-04-01 01:30 it's removed 2008-04-01 01:30 I will open an experimental branch tomorrow 2008-04-01 01:31 Actually, that's weird. The sources were removed, but the Makefile wasn't updated. 2008-04-01 01:31 with that cleanup among other things in it 2008-04-01 01:31 yes, odd 2008-04-01 01:31 Don't look at me man, I just tried to clean up the broken dependencies and such. 2008-04-01 01:31 I wasn't looking at you 2008-04-01 01:31 a makefile is capable of stinking all on its own 2008-04-01 01:32 It's weird. For such a small makefile, it has a large amount of rot, eh? 2008-04-01 01:32 yes 2008-04-01 01:32 small, but long lived 2008-04-01 01:32 I blame not using enough make tricks, but you know that. ;-) 2008-04-01 01:33 imagine how it would look now with 5 years of accumated makefile tricks 2008-04-01 01:33 it did prove robust 2008-04-01 01:33 lol 2008-04-01 01:34 I am now running ddtest, which provides a stub main that does not include ddsnap.c, but does include ddsnapd.c 2008-04-01 01:34 yup, that's how it should be done. 2008-04-01 01:35 there's no fancy formalism here, but it's better than it was 2008-04-01 01:35 It'd be nice if it were easier to stub out ddsnap.c functions though. 2008-04-01 01:35 ddsnap.c could use some organizational lovin 2008-04-01 01:35 but first it just needs a whole lot of cut n paste reduction 2008-04-01 01:35 OH YEAH! 2008-04-01 01:36 / begin BOGUS KILL ME !!! <- new code marker for init_change_list and unholy friends 2008-04-01 01:36 lol 2008-04-01 01:37 the next 3 months or so really needs to be split evenly between cleanup and optimization 2008-04-01 01:39 $ ./ddtest /src/zuma/dev1 /src/zuma/dev2 2008-04-01 01:39 Tue Apr 1 01:38:23 2008: [2929] change_device_sizes: metadev size changes from 0 to 2500 2008-04-01 01:39 Tue Apr 1 01:38:23 2008: [2929] change_device_sizes: snapdev size changes from 0 to 2500 2008-04-01 01:39 Tue Apr 1 01:38:23 2008: [2929] change_device_sizes: orgdev size changes from 0 to 20000 2008-04-01 01:39 Tue Apr 1 01:38:23 2008: [2929] init_allocation: metadata store size: 2500 chunks (20000 sectors) 2008-04-01 01:39 Tue Apr 1 01:38:23 2008: [2929] init_allocation: snapshot store size: 2500 chunks (20000 sectors) 2008-04-01 01:39 Tue Apr 1 01:38:23 2008: [2929] init_bitmap_blocks: Initializing 1 bitmap block(s) 2008-04-01 01:39 Tue Apr 1 01:38:23 2008: [2929] init_bitmap_blocks: Initializing 1 bitmap block(s) 2008-04-01 01:39 some wrong factoring in that change_device_sizes thing 2008-04-01 01:39 need to go over that with jiaying 2008-04-01 01:40 hmm 2008-04-01 01:41 works well enough for the immediate purpose 2008-04-01 01:41 now to unit test journal replay 2008-04-01 01:48 wow, the old unit tests still work 2008-04-01 01:48 now that is resilience 2008-04-01 01:49 oh, with the journal rewrite too 2008-04-01 01:49 journal replay rewrite 2008-04-01 01:49 well, that's how it should be 2008-04-01 01:49 the code hasn't even been compiled for four years 2008-04-01 01:50 i think it was flipped on during some debugging 2008-04-01 01:50 hasn't changed either though, right? 2008-04-01 01:50 yes it has 2008-04-01 01:50 see "with the journal replay rewrite" above 2008-04-01 01:50 who changes code w/o compiling it? 2008-04-01 01:50 the code didn't change, everything around it did 2008-04-01 01:51 oh that stuff, i c 2008-04-01 02:34 wow, just spent an hour finding out why free count stat doesn't match bitmap in the unit test, and it was because I was... um... deferring the allocation :-] 2008-04-01 02:34 which is rather the point of this 2008-04-01 02:34 at least we know unit tests are doing something 2008-04-01 02:37 heh 2008-04-01 02:40 you're up too late ;) 2008-04-01 02:43 you arent? 2008-04-01 02:43 ;) 2008-04-01 02:47 unit test found a bug 2008-04-01 02:47 - return count + deferred; 2008-04-01 02:47 + return count - deferred; 2008-04-01 02:48 ok, that's enough for today 2008-04-01 02:48 I can see getting through the worst of this optimization damage tomorrow 2008-04-01 02:52 git is actually infuriating for somebody developing a patch as opposed to merging 2008-04-01 02:53 after you change a bunch of files, you then have to tell git "yes I really meant to change those files, do commit them when I say commit" 2008-04-01 02:53 this alone is worth using mercurial instead of git 2008-04-01 02:53 unless you are a kernel dev and have no choice 2008-04-01 02:54 I think I will change my zumastor repo from git to mercurial, and leave the ddsnap/lvm3 repo as git 2008-04-01 02:54 will sleep on that 2008-04-01 03:07 or create an alias? 2008-04-01 03:07 and doesn't commit -a do that? 2008-04-01 03:08 it does 2008-04-01 03:08 I knew that, and forgot 2008-04-01 03:08 this aspect of git is arguably the worst blemish 2008-04-01 03:08 and its more than a blemish 2008-04-01 03:08 it is a gigantic festering wound 2008-04-01 03:09 zumastor still moves to mercurial I think 2008-04-01 03:11 what annoys me more than svn 2008-04-01 03:11 is switching between version control systems for projects 2008-04-01 03:12 an argument for using git for everything? 2008-04-01 03:12 http://phunq.net/ddtree?p=zumastor/.git;a=summary <- snapshot of tonight's work 2008-04-01 03:13 i suppose... and work on improving git? 2008-04-01 03:13 I think it's beyond hope 2008-04-01 03:13 people have tried to explain why that index concept is bad 2008-04-01 03:13 mercurial proves it is unnecessary 2008-04-01 03:14 but gitsters like their wart 2008-04-01 03:14 hrm looking at your gitweb change 2008-04-01 03:14 even try to present it as a feature 2008-04-01 03:14 you have a lot of stuff in that one commit 2008-04-01 03:14 true 2008-04-01 03:14 it's going to be like that for this patch I am afraid 2008-04-01 03:14 didn't even separate out the aio stuff on journal stuff 2008-04-01 03:14 what aio stuff? 2008-04-01 03:14 there should be none 2008-04-01 03:15 er i mean unit test 2008-04-01 03:15 right, it will be one glob 2008-04-01 03:15 i suppose 2008-04-01 03:15 factoring is too braindamaged to make a clean separation practical 2008-04-01 03:16 see, it's not possible to know what the optimization commit should be without at least some unit testing 2008-04-01 03:17 it would be wanking to try to get it exactly right without actually trying it first 2008-04-01 03:17 and by that time, the unit stuff is all in and would be work to remove 2008-04-01 03:19 -!- MaZe(~MaZe@193.120.148.177) has joined #zumastor 2008-04-01 03:51 -!- phoenix24(cqkzf@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-01 03:53 zumastor-commits: [zumastor commit] r1510 - www 2008-04-01 03:58 zumastor-commits: [zumastor commit] r1511 - www 2008-04-01 04:01 -!- pgquiles_(~pgquiles@62.43.226.52.static.user.ono.com) has joined #zumastor 2008-04-01 04:03 zumastor-commits: [zumastor commit] r1512 - www 2008-04-01 04:15 Hi everyone! 2008-04-01 04:18 Is the code repo already being hosted other than svn ? 2008-04-01 04:33 -!- pgquiles__(~pgquiles@226.Red-88-22-55.staticIP.rima-tde.net) has joined #zumastor 2008-04-01 04:41 hi phoenix24 2008-04-01 04:41 it is currently in svn, but daniel phillips has his own git tree he uses for development 2008-04-01 04:57 -!- erwan_taf(~erwan@81.80.43.67) has joined #zumastor 2008-04-01 05:27 -!- phoenix24(ppotui@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-01 06:28 -!- phoenix24(pemma@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-01 06:28 -!- MaZe(~MaZe@193.120.148.177) has joined #zumastor 2008-04-01 07:32 -!- phoenix24(qljlpp@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-01 09:25 ACTION fixes his switch and goes to work 2008-04-01 09:29 -!- cbsmith(~xman@adsl-99-165-23-73.dsl.lsan03.sbcglobal.net) has joined #zumastor 2008-04-01 09:54 -!- jiayingz(~jiayingz@207.47.98.129.static.nextweb.net) has joined #zumastor 2008-04-01 10:58 -!- tim_vimm(~Tim@cpe-76-90-128-140.socal.res.rr.com) has joined #zumastor 2008-04-01 11:19 -!- tim_vimm(~Tim@cpe-76-90-128-140.socal.res.rr.com) has joined #zumastor 2008-04-01 12:10 zumastor-commits: [zumastor commit] r1513 - in trunk: . zumastor/debian 2008-04-01 12:14 zumastor-commits: [PPA zumastor-team] Accepted: zumastor 0.8.0-r1513 (source) 2008-04-01 12:14 zumastor-commits: [PPA zumastor-team] Accepted: ddsnap 0.8.0-r1513 (source) 2008-04-01 12:59 I had to request another G of storage on our ppa 2008-04-01 13:02 -!- cbsmith(~xman@207.47.98.129.static.nextweb.net) has joined #zumastor 2008-04-01 13:07 -!- mitchg(~mitchg@207.47.98.129.static.nextweb.net) has joined #zumastor 2008-04-01 13:15 jiayingz: Did that modified patch look ok? 2008-04-01 13:24 willn, yes 2008-04-01 13:25 willn, are you running the uml smoke test? 2008-04-01 13:25 im building a .deb to test on real hardware 2008-04-01 13:37 -!- mitchg(~mitchg@207.47.98.129.static.nextweb.net) has joined #zumastor 2008-04-01 13:40 zumastor-commits: [zumastor commit] r1514 - trunk/cbtb/host-scripts 2008-04-01 13:55 zumastor-commits: [zumastor commit] r1515 - trunk/kernel/config 2008-04-01 14:15 -!- tim_vimm(~Tim@cpe-76-90-128-140.socal.res.rr.com) has joined #zumastor 2008-04-01 14:45 zumastor-commits: [PPA zumastor-team] Accepted: linux 2.6.24-13.23~zumappa2 (source) 2008-04-01 14:47 ping flips 2008-04-01 14:50 -!- tim_vimm(~Tim@cpe-76-90-128-140.socal.res.rr.com) has joined #zumastor 2008-04-01 15:23 -!- tim_vimm(~Tim@cpe-76-90-128-140.socal.res.rr.com) has joined #zumastor 2008-04-01 15:40 zumastor-commits: [zumastor commit] r1516 - trunk/ddsnap/patches/2.6.24.2 2008-04-01 15:45 zumastor-commits: [zumastor commit] r1517 - trunk 2008-04-01 16:47 jiayingz, hi 2008-04-01 17:32 http://phunq.net/ddtree?p=zumastor/.git;a=summary 2008-04-01 17:33 shapor, how do I set up an autodribble of ddtree/zumastor commits to the channel? 2008-04-01 17:33 some rss cuteness? 2008-04-01 17:33 or willn? 2008-04-01 17:36 the gitweb does appear to provide an rss feed 2008-04-01 17:36 could feed that in to email, which could use procmail to talk to an irc bot 2008-04-01 17:36 could even make the zumalog bot do it 2008-04-01 17:36 would be pretty easy 2008-04-01 17:38 I'm _completely_ clueless about procmail 2008-04-01 17:38 but I could give it a try 2008-04-01 17:38 or even easier 2008-04-01 17:38 sounds good... 2008-04-01 17:38 something polling the rss 2008-04-01 17:38 even a shell script 2008-04-01 17:38 like a bot? ;-) 2008-04-01 17:38 calling curl or something gross like that 2008-04-01 17:38 but not a perl bot 2008-04-01 17:38 bot would be fine 2008-04-01 17:38 just not a perl bot 2008-04-01 17:39 the bot part we already have 2008-04-01 17:39 all you have to do is echo to it to write to the channel on the zumastor.org server 2008-04-01 17:39 isn't that like 3 lines? 2008-04-01 17:39 to add to the bot? 2008-04-01 17:39 hi all 2008-04-01 17:39 hi zumalog 2008-04-01 17:40 echo hi all > ii/irc.oftc.net/#zumastor/in 2008-04-01 17:40 ACTION knows it's getting near the end when he starts talking to teh_ bots 2008-04-01 17:40 pipes :) 2008-04-01 17:40 ic 2008-04-01 17:40 kool 2008-04-01 17:41 the only tricky part is keeping track of the last change 2008-04-01 17:41 so it only sends new things 2008-04-01 17:41 unfortunate that rss isn't actually a stream 2008-04-01 17:45 willn-colo: rss-add ddt-zumastor http://phunq.net/ddtree?p=zumastor/.git;a=rss 2008-04-01 17:45 rss item added 2008-04-01 17:45 willn-colo: rss-watch ddt-zumastor 2008-04-01 17:45 watcher thread started 2008-04-01 17:45 boo 2008-04-01 17:45 so much less cool 2008-04-01 17:45 :P 2008-04-01 17:45 2 lines, 5 seconds 2008-04-01 17:45 heh 2008-04-01 17:48 i've no idea what interval it will poll 2008-04-01 17:48 5 mins i think 2008-04-01 18:33 -!- phoenix24(xti@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-01 19:20 -!- phoenix24(ejxxr@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-01 19:20 Hi 2008-04-01 19:45 hi phoenix24 2008-04-01 19:46 let's try it 2008-04-01 19:46 new checkin coming in 10-15 minutes 2008-04-01 19:46 try ? 2008-04-01 19:46 rss feed from the zumastor git repo 2008-04-01 19:46 http://phunq.net/zumastor 2008-04-01 19:47 E:404 URL Not Found! 2008-04-01 19:47 http://phunq.net/ddtree sorry! 2008-04-01 19:48 and http://phunq.net/git/zumastor for cloning/pulling 2008-04-01 19:51 Fails too! 2008-04-01 19:52 which one fails? 2008-04-01 19:52 http://phunq.net/git/zumastor, fails while attempting to clone 2008-04-01 19:53 may I see your git clone command? 2008-04-01 19:54 got it!! 2008-04-01 19:54 good :) 2008-04-01 19:55 It seems I've caused an identity crisis for myself on the mailing list. 2008-04-01 19:57 Chaitanya Sharma (legal name), phoenix24 (irc). 2008-04-01 19:59 no problem 2008-04-01 20:01 regarding the SoC application, I'm revising it again. Other than that, where must I add further details or make changes ? 2008-04-01 20:02 let me read it again 2008-04-01 20:02 in a couple minutes 2008-04-01 20:03 On repository, shall we start using "git" for all purposes and not svn ? 2008-04-01 20:03 svn is authoritative for the time being 2008-04-01 20:03 ok 2008-04-01 20:03 feel free to operate your own git repo though 2008-04-01 20:04 you can pull from me and create your own branch 2008-04-01 20:04 recommended 2008-04-01 20:04 nice 2008-04-01 20:04 it is :) 2008-04-01 20:05 flips you going to check something in to your git soon 2008-04-01 20:05 i'm dying to see if my oneliner works 2008-04-01 20:05 soon, yes 2008-04-01 20:05 ;) 2008-04-01 20:05 just about ready 2008-04-01 20:05 completely untested ;) 2008-04-01 20:05 hi shapor! 2008-04-01 20:05 hi phoenix24 2008-04-01 20:06 flips you might notice lots of the HEAD requests on your rss feed ;) 2008-04-01 20:06 I need to start noticing 2008-04-01 20:11 just pulled the latest private hack into public repo 2008-04-01 20:11 ACTION starts counting 2008-04-01 20:11 shapor, event-driveh = good; polled = kinda ok :-) 2008-04-01 20:12 -!- phoenix24_(afmtp@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-01 20:16 ddt-zumastor: It is necessary to remember the deferred allocations in some way so 2008-04-01 20:17 wihhn, suhweet 2008-04-01 20:17 willn even 2008-04-01 20:23 < http://phunq.net/ddtree?p=zumastor/.git;a=commit;h=a370460c6c59a782f47bdc528052dcb4d5b64c00 2008-04-01 20:23 < It is necessary to remember the deferred allocations in some way so 2008-04-01 20:23 :) 2008-04-01 20:24 boo 2008-04-01 20:24 didn't trim it right :( 2008-04-01 20:25 but the info came in 2008-04-01 20:25 yeah 2008-04-01 20:25 the idea is there 2008-04-01 20:25 i'll save the command 2008-04-01 20:25 can you show just the last 8 digits of the object hash? 2008-04-01 20:25 yeah, its bash 2008-04-01 20:25 or 4 or 6 even 2008-04-01 20:25 heh 2008-04-01 20:25 no need to duplicate willn-colo 2008-04-01 20:25 patent attorneys rely on 3 2008-04-01 20:26 just another part of the whole braindamaged patent racket 2008-04-01 20:26 I must learn this magic 2008-04-01 20:26 the scripty stuff 2008-04-01 20:34 the reason why it wasn't working is that the gitweb cgi doesn't set the last-modified http header 2008-04-01 20:35 so i dont know when the rss feed is updated without completely re-downloading 2008-04-01 20:35 which sucks cause its like 180k 2008-04-01 20:35 just install git + gitweb, hack the script to fix it and send me the diff? 2008-04-01 20:35 i suppose i could just stop after i've read the relevent xml, but still, thats lameness 2008-04-01 20:35 I'll talk you through repo setup ;) 2008-04-01 20:36 i have already done that 2008-04-01 20:36 let's fix gitweb's html 2008-04-01 20:36 (setup gitweb for another repo) 2008-04-01 20:36 did you pull zumastor from me? 2008-04-01 20:36 yeah last night 2008-04-01 20:36 i'm not using it though 2008-04-01 20:36 so when is your legendary hack landing? 2008-04-01 20:37 hardly legendary 2008-04-01 20:37 not before it lands, no 2008-04-01 20:37 hah 2008-04-01 20:37 just a bugfix really 2008-04-01 20:37 should use the timestamp of the most recent change 2008-04-01 20:37 well then when is a legendary hack coming? 2008-04-01 20:37 what is the use of having a git repo without having legendary hacks in it? 2008-04-01 20:38 i suppose since you're persistent i will take a look at it now 2008-04-01 20:38 ;) 2008-04-01 20:38 mmm 5183 lines of perl 2008-04-01 20:39 woot 2008-04-01 20:39 i'm glad its all in one file 2008-04-01 20:39 mercurial wins again 2008-04-01 20:39 no bogus factoring out :P 2008-04-01 20:39 except gitweb is nicer looking and more comprehensive than mercurial web 2008-04-01 20:41 ah it does set the last-modified 2008-04-01 20:41 however, only for a GET 2008-04-01 20:41 HEAD returns: 2008-04-01 20:41 Last-Modified: Thu, 01 Jan 1970 00:00:00 GMT 2008-04-01 20:41 so gitweb is lame & not u?> 2008-04-01 20:42 4956 return if ($cgi->request_method() eq 'HEAD'); 2008-04-01 20:42 hrm 2008-04-01 20:42 looks like a bug 2008-04-01 20:42 NEWS FLASH: msooxml approved on april fool's day 2008-04-01 20:43 "return if" is a hideous construct 2008-04-01 20:43 Apr 2: EU antitrust investigation begins 2008-04-01 20:43 MS will pay more for that bought vote than they intended 2008-04-01 20:44 flips: what version of git-web are you running? 2008-04-01 20:44 "Summer of Code Deadline Extended 6 Days" http://google-opensource.blogspot.com/2008/03/one-more-week-to-apply-to-google-summer.html 2008-04-01 20:45 gitweb 1.5.4.2 2008-04-01 20:45 leet version 2008-04-01 20:45 hrm this is 1.5.2.5 2008-04-01 20:46 ddt-zumastor: ...and also test for within the (soon to die) sb alloc/alloced range 2008-04-01 20:46 you know its not even setting the last modified right for GET 2008-04-01 20:46 improperly setting last-modified to epoch is a bad bug 2008-04-01 20:47 will fend off crawlers and properly written rss feed readers 2008-04-01 20:48 a bug?? 2008-04-01 20:48 say it ain't so 2008-04-01 20:49 hrm i'm running it on to pof boa and i get no last-modified header 2008-04-01 20:50 i guess it doesn't support that bit of the cgi interface 2008-04-01 20:50 time to switch to apache 2008-04-01 20:51 hrm that cant be 2008-04-01 20:52 hrm its because i have no commits 2008-04-01 20:53 why wouldnt they be getting published to get-web? 2008-04-01 20:53 s/get-/git-/ 2008-04-01 20:57 ah permissions, duh 2008-04-01 20:58 flips: hey it sets it properly in my version 2008-04-01 20:58 yours is broken 2008-04-01 20:59 hrmph 2008-04-01 20:59 my leet version? 2008-04-01 20:59 ACTION thinks higher numbers are better numbers 2008-04-01 21:00 did I break it by adding a penquin picture? 2008-04-01 21:01 hah, no 2008-04-01 21:01 is it straight from upstream tgz? 2008-04-01 21:03 um 2008-04-01 21:03 from debian backports.org 2008-04-01 21:03 ahh 2008-04-01 21:03 can you just send me your gitweb.cgi then? 2008-04-01 21:03 ACTION is running stable :-o 2008-04-01 21:16 ddt-zumastor: change sb alloc/alloced variables to struct alloc_range same as the vector 2008-04-01 21:16 ok, about time to stop improving and test this hack deeper 2008-04-01 21:20 i'm going to copy yours so i can diff it against a working one 2008-04-01 21:27 flips: hah you hacked it up i see 2008-04-01 21:27 1060 + print "\n" ; 2008-04-01 21:30 yah 2008-04-01 21:30 figured it needed a penguin 2008-04-01 21:30 hrm theres a lot of stuff changed between this one and the one i have which works 2008-04-01 21:30 i dont feel like bisecting it 2008-04-01 21:31 report a bug? 2008-04-01 21:31 first why dont you try the latest gitweb.cgi 2008-04-01 21:31 although hrm, debianhas probably hacked some paths in to it 2008-04-01 21:32 is it installed from apt on your server? 2008-04-01 21:32 it is 2008-04-01 21:32 I'll need a diff of my hacks... 2008-04-01 21:32 or not 2008-04-01 21:32 hmm 2008-04-01 21:32 just grab the tux line 2008-04-01 21:34 is my bot working out fo ryou 2008-04-01 21:35 willn, the bot works fine 2008-04-01 21:35 it is.. mine only failed do to a gitweb bug ;) 2008-04-01 21:36 see above commit massages 2008-04-01 21:36 mine was one line of shell ;) 2008-04-01 21:36 let's see if backports has a better version of gitweb for me 2008-04-01 21:37 otherwise I will just get it from gitweb repo wherever 2008-04-01 21:37 damn ppa build failed 2008-04-01 21:37 just try a newer .cgi from a tarball maybe 2008-04-01 21:37 arg 2008-04-01 21:37 that's as new as debian gets 2008-04-01 21:37 where's the home repo? 2008-04-01 21:37 ACTION clicks on the icon 2008-04-01 21:54 By design, Zumastor can theoretically take backups over any native FS. 2008-04-01 21:54 -!- phoenix24(vbbm@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-01 21:56 Would it be a good idea to design Zumastor itself a native FS ? 2008-04-01 21:57 big project 2008-04-01 21:57 there is already btrfs underway 2008-04-01 21:57 yes 2008-04-01 21:57 when btrfs is ready, we will support it in zumastor 2008-04-01 21:58 we could start trying it out pretty soon in fact 2008-04-01 21:58 but, would btrfs need an external back-up utility on top of it ? 2008-04-01 21:58 it would hopefully provide the same snapshot facility as ddsnap 2008-04-01 21:58 exactly, my point 2008-04-01 21:59 we would have to code replication for it, unless the oracle guys already have done it 2008-04-01 22:11 Zumastor doesn't have a API as of now, which could contain certain wrapper functionalities. 2008-04-01 22:12 such as for resizing, status, replay.. etc. 2008-04-01 22:12 and while adding resize support to LVM2, it would be useful. 2008-04-01 22:12 the api is essentially the cli 2008-04-01 22:13 but API would also help in making tiny binaries each for specific functionality. 2008-04-01 22:13 rather than contain everything in one binary - ddsnap 2008-04-01 22:14 having one binary is far from our largest problem right now 2008-04-01 22:14 zfs has one binary.. people seem to like that 2008-04-01 22:14 ture 2008-04-01 22:15 ZFS.. Zettabyte FS ? 2008-04-01 22:15 yes 2008-04-01 22:15 zfs foo 2008-04-01 22:15 yes 2008-04-01 22:15 not hard to simulate multiple binaries with hardlinks if you really wanted 2008-04-01 22:16 ddsnap-transmit ddsnap-create, etc 2008-04-01 22:16 yeah 2008-04-01 22:16 the ddsnap cli parsing code is quite ugly right now 2008-04-01 22:17 putting a stick of dynamite in the center of it and exploding it in to lots of little messes doesn't seem that appealing to me ;) 2008-04-01 22:17 yeah, could use.. glic's option parsing. 2008-04-01 22:17 one of these days someone will put a couple days in to it and make it pretty 2008-04-01 22:17 :) 2008-04-01 22:26 id love to, but im restricted to testing =p 2008-04-01 22:28 willn says who 2008-04-01 22:29 I could give a shot ? 2008-04-01 22:30 ACTION is off to lecture! 2008-04-01 22:31 shapor: I signed on with it, gotta follow through 2008-04-01 22:34 yeah but you can always do extra :) 2008-04-01 22:35 i work on zumastor mostly on my own time now, but i'm stupid :P 2008-04-01 22:41 heh 2008-04-01 22:41 i've got a friend coming into town for all of next week, so im tring to be super productive this week 2008-04-01 22:55 turns out running the cbtb tests on real hardware is a pain in the neck 2008-04-01 22:59 whys that 2008-04-01 23:09 they've all got hardcoded devices (sdb, sdc) for testing on 2008-04-01 23:12 so, todo for tommorow: 2008-04-01 23:12 Get all of cbtb running under autotest 2008-04-01 23:13 Hook into Host.machine_install with the installer i wrote 2008-04-01 23:14 The former should be doable 2008-04-01 23:15 the latter may run the rest of the week 2008-04-01 23:37 zumastor-commits: [zumastor commit] r1518 - trunk/cbtb/tests/1 2008-04-01 23:38 -!- phoenix24(dpgxvdx@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-01 23:39 What do we think about having all tests run on lvm 2008-04-01 23:40 that how we used to do it 2008-04-01 23:40 i thought 2008-04-01 23:40 i'm in favor 2008-04-01 23:41 essentially all our tests need to be hermetic 2008-04-01 23:41 yeah 2008-04-01 23:41 but id like to start them off with a similiar base and configure lvm as the test demands 2008-04-01 23:46 phoenix, a bigger issue re api is whether btrfs has one for replication 2008-04-01 23:47 I suppose I should explain what they need to them 2008-04-01 23:56 Suppose I should sleep so I can get this done tommorow. irc.oftc.net #zumastor log beginning Wed Apr 2 00:00:01 PDT 2008 2008-04-02 00:08 flips: I have'nt lookrd at brtfs src, lemme see. I'd let you know. 2008-04-02 00:39 -!- phoenix24(ghtpgee@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-02 01:28 -!- pgquiles_(~pgquiles@62.43.226.52.static.user.ono.com) has joined #zumastor 2008-04-02 01:34 -!- pgquiles__(~pgquiles@226.Red-88-22-55.staticIP.rima-tde.net) has joined #zumastor 2008-04-02 01:42 -!- phoenix24(ffmgdho@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-02 02:20 -!- pgquiles_(~pgquiles@62.43.226.52.static.user.ono.com) has joined #zumastor 2008-04-02 02:25 -!- pgquiles__(~pgquiles@226.Red-88-22-55.staticIP.rima-tde.net) has joined #zumastor 2008-04-02 03:04 -!- phoenix24(dkiha@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-02 04:07 -!- phoenix24(sonl@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-02 05:08 -!- phoenix24_(wsvicfl@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-02 06:08 -!- phoenix24(myhx@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-02 09:29 -!- cbsmith(~xman@adsl-99-165-23-73.dsl.lsan03.sbcglobal.net) has joined #zumastor 2008-04-02 09:33 -!- phoenix24(vxvtksl@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-02 09:50 -!- erwan_taf(~erwan@81.80.43.67) has joined #zumastor 2008-04-02 10:21 positive reinforcement from the autotest folks, im happy 2008-04-02 10:55 -!- phoenix24(nrghqqm@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-02 12:13 zumastor-commits: [zumastor commit] r1519 - in trunk: cbtb/host-scripts kernel/config 2008-04-02 12:25 -!- phoenix24(qqa@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-02 13:07 -!- phoenix24(hpquuc@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-02 13:38 -!- willn(~wan@pinball.ccs.neu.edu) has joined #zumastor 2008-04-02 13:38 -!- willn-colo(wan@209.67.252.126) has joined #zumastor 2008-04-02 14:08 -!- phoenix24(mwcalee@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-02 14:41 the delta's generated between two snapshots, already supports extents 2008-04-02 14:41 am i correct here? 2008-04-02 14:43 and a chuck ___is___ the data unit addressable at the lowest level. 2008-04-02 14:52 Why use posix_memalign() ? 2008-04-02 15:08 -!- phoenix24_(qtmenj@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-02 15:39 zumastor-commits: [zumastor commit] r1520 - trunk/ddsnap/man 2008-04-02 15:53 phoenix24_: 'cause O_DIRECT requires aligned memory. 2008-04-02 15:54 ok 2008-04-02 15:56 cbsmith: field "sb->image.journal_size = js_chunks" refers to the journal size in terms of #NOS of chunks 2008-04-02 15:56 rather than, chunk*sizeof(chunk). 2008-04-02 16:29 -!- phoenix24(dzgvi@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-02 16:31 -!- cbsmith(~xman@adsl-99-165-23-73.dsl.lsan03.sbcglobal.net) has joined #zumastor 2008-04-02 16:45 zumastor-commits: [zumastor commit] r1521 - trunk/cbtb/host-scripts 2008-04-02 17:55 -!- phoenix24(almxv@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-02 19:24 jiayingz: I'm going to head home, email me if you need anything else 2008-04-02 19:33 willn, thanks! 2008-04-02 19:56 -!- tleuser_(~tleuser@ftp.panithatyai.ac.th) has joined #zumastor 2008-04-02 20:01 -!- phoenix24(aliw@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-02 21:01 -!- cbsmith(~xman@adsl-99-165-23-73.dsl.lsan03.sbcglobal.net) has joined #zumastor 2008-04-02 21:01 -!- mitchg(~mitchg@207.47.98.129.static.nextweb.net) has joined #zumastor 2008-04-02 21:01 -!- jiayingz(~jiayingz@207.47.98.129.static.nextweb.net) has joined #zumastor 2008-04-02 21:01 -!- natalie(~nataliep@207.47.98.129.static.nextweb.net) has joined #zumastor 2008-04-02 21:01 -!- flips(~phillips@phunq.net) has joined #zumastor 2008-04-02 21:01 -!- dld(~dld@207.47.98.129.static.nextweb.net) has joined #zumastor 2008-04-02 21:24 -!- dld(~dld@207.47.98.129.static.nextweb.net) has joined #zumastor 2008-04-02 21:24 -!- flips(~phillips@phunq.net) has joined #zumastor 2008-04-02 21:24 -!- natalie(~nataliep@207.47.98.129.static.nextweb.net) has joined #zumastor 2008-04-02 21:24 -!- jiayingz(~jiayingz@207.47.98.129.static.nextweb.net) has joined #zumastor 2008-04-02 21:24 -!- mitchg(~mitchg@207.47.98.129.static.nextweb.net) has joined #zumastor 2008-04-02 21:24 -!- cbsmith(~xman@adsl-99-165-23-73.dsl.lsan03.sbcglobal.net) has joined #zumastor 2008-04-02 21:24 -!- willn-colo(wan@209.67.252.126) has joined #zumastor 2008-04-02 21:24 -!- willn(~wan@pinball.ccs.neu.edu) has joined #zumastor 2008-04-02 21:24 -!- erwan_taf(~erwan@81.80.43.67) has joined #zumastor 2008-04-02 21:24 -!- pgquiles__(~pgquiles@226.Red-88-22-55.staticIP.rima-tde.net) has joined #zumastor 2008-04-02 21:36 -!- phoenix24(miwkkj@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-02 22:08 hooray for things that work 2008-04-02 22:09 :) 2008-04-02 22:09 really 2008-04-02 22:09 what worked? 2008-04-02 22:11 Hi! 2008-04-02 22:11 hi phoenix24 2008-04-02 22:12 I have to go check some proposals 2008-04-02 22:12 SoC proposals ? 2008-04-02 22:12 yes 2008-04-02 22:15 The delta's generated between two snapshots, already supports extents ? 2008-04-02 22:32 yes 2008-04-02 22:32 that code is pretty ugly too, ugh 2008-04-02 22:33 nautilus... I don't think I want that (proposal to provide previous versions interface via nautilus plugin) 2008-04-02 22:33 firefox plugin would be groovy 2008-04-02 22:33 comments? 2008-04-02 22:36 -!- phoenix24_(vjoj@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-02 22:40 flips: autotest 2008-04-02 23:16 I'm having some confusion with terms here: 2008-04-02 23:16 allocsize_*, wherever mentioned; always refers to the to actual allocation size. 2008-04-02 23:17 and a chuck _is_ the smallest addressable data unit. (at the lowest level) 2008-04-02 23:37 -!- phoenix24(oaqplnu@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-02 23:39 willn, sounds groovE 2008-04-02 23:40 phoenix24, chunks is variable size, sorry that we have that allocsize terminology for the same thing, it needs to be cleaned up 2008-04-02 23:40 next week ok? 2008-04-02 23:56 flips: next week for cleanup ? irc.oftc.net #zumastor log beginning Thu Apr 3 00:00:01 PDT 2008 2008-04-03 00:00 yes 2008-04-03 00:05 flips: absolutely fine 2008-04-03 00:16 so we have three proposals to provide previous versions guis and two to add extents, one of which proposes to tackle online resizing first 2008-04-03 00:16 wow 2008-04-03 00:17 I want everybody to work on all of it ;-) 2008-04-03 00:17 but I think I am expected to pick some subset :( 2008-04-03 00:38 -!- phoenix24(xcqs@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-03 00:38 ACTION goes to sleep so he can get 2 machine tests and 2.6.24 builds working tommorow 2008-04-03 00:40 *yawn* 2008-04-03 00:40 ACTION better do something about this sleep thing 2008-04-03 00:43 flips: I'll be re-submitting my proposal, in an effort to make a strong case technically (& otherwise) on my behalf. 2008-04-03 00:44 correcting certain mistake I made and add relevant details at appropriate places. 2008-04-03 01:38 -!- phoenix24_(jbe@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-03 02:39 -!- phoenix24(pxraq@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-03 02:41 flips ping 2008-04-03 02:42 shapor, pong 2008-04-03 02:42 looking at ddsnapd.. squahsing doesn't add much code 2008-04-03 02:42 phoenix24, let me know when you do 2008-04-03 02:42 not a lot indeed 2008-04-03 02:42 ~15 lines 2008-04-03 02:43 still we should do it properly... first add decent error handling, then remove squashing 2008-04-03 02:43 flips: I'll send it to the mailing list.. for a feedback first; and definitely inform you. 2008-04-03 02:43 good idea 2008-04-03 02:44 shapor, and maybe think about getting some sleep? 2008-04-03 02:45 whats that? 2008-04-03 02:45 it's what makes you shoot straight in 8 ball 2008-04-03 02:46 i think i'm going to do a git tree as well 2008-04-03 02:46 so much nicer than mailing patches 2008-04-03 02:46 indeed 2008-04-03 02:46 from scratch, or clone from me? 2008-04-03 02:47 neat thing is, either way you get the same objects because each unique file text has a globally unique hash 2008-04-03 02:48 probably just pull from you, its one less command 2008-04-03 02:49 git is all about everybody pulling from everybody 2008-04-03 03:01 flips: see mailed patch to the list 2008-04-03 03:05 flips what happens when the server dies during a delete? 2008-04-03 03:05 we Compress the snapshot entry out of the snapshot list 2008-04-03 03:06 and then do the tree delete 2008-04-03 03:08 tomorrow... 2008-04-03 03:10 oh i see, we are just setting it dirty 2008-04-03 03:10 and write it out when we are done deleting, duh 2008-04-03 03:11 haven't look at this in a while 2008-04-03 03:11 and its late 2008-04-03 03:21 "sb->copybuf ", its used for copy-on-write; isn't it ? 2008-04-03 03:37 yes iirc 2008-04-03 03:39 -!- phoenix24_(vwzvz@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-03 03:40 its also used as a copy buffer for bitmap updates as well 2008-04-03 03:56 How is the calculation done ? 2008-04-03 03:56 sb->image.metadata.freechunks += metachunks - sb->image.metadata.chunks + oldbitmaps - sb->image.metadata.bitmap_blocks; 2008-04-03 03:59 "sb->max_commit_blocks", What is this field ? 2008-04-03 04:39 -!- phoenix24(azepb@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-03 05:29 -!- phoenix24(azepb@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-03 05:29 -!- dld(~dld@207.47.98.129.static.nextweb.net) has joined #zumastor 2008-04-03 05:29 -!- flips(~phillips@phunq.net) has joined #zumastor 2008-04-03 05:29 -!- natalie(~nataliep@207.47.98.129.static.nextweb.net) has joined #zumastor 2008-04-03 05:29 -!- jiayingz(~jiayingz@207.47.98.129.static.nextweb.net) has joined #zumastor 2008-04-03 05:29 -!- mitchg(~mitchg@207.47.98.129.static.nextweb.net) has joined #zumastor 2008-04-03 05:29 -!- willn-colo(wan@209.67.252.126) has joined #zumastor 2008-04-03 05:29 -!- willn(~wan@pinball.ccs.neu.edu) has joined #zumastor 2008-04-03 05:29 -!- erwan_taf(~erwan@81.80.43.67) has joined #zumastor 2008-04-03 05:29 -!- pgquiles__(~pgquiles@226.Red-88-22-55.staticIP.rima-tde.net) has joined #zumastor 2008-04-03 09:17 shapor, ->copybuf is not used as a copy buffer for bitmap updates, only for copy-before-write aka copyout 2008-04-03 09:21 phoenix24 (when you come back) ->max_commit_blocks is the number of journal blocks that can be included in a single journal commit, which is limited by the space available in the commit block for recording the physical addresses of the committed blocks 2008-04-03 10:22 -!- tim_vimm(~Tim@cpe-76-90-128-140.socal.res.rr.com) has joined #zumastor 2008-04-03 10:30 -!- phoenix24(dwiwqgp@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-03 10:30 -!- phoenix24_(pcckoye@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-03 10:37 flips: in update_bitmap: 2008-04-03 10:37 3004 diskread(sb->metadev, sb->copybuf, blocksize, oldbase + (i << blockshift)); 2008-04-03 10:37 3005 diskwrite(sb->metadev, sb->copybuf, blocksize, newbase + (i << blockshift)); 2008-04-03 10:38 ah, new code 2008-04-03 10:38 fair enough 2008-04-03 10:39 this is not bitmap update per se 2008-04-03 10:39 but part of resize 2008-04-03 10:39 need to fix the function name 2008-04-03 10:40 yeah i realized that after 2008-04-03 10:40 -static int update_bitmap(struct superblock *sb, struct allocspace *as, u64 newchunks, u64 new_basechunk, u64 new_metachunks) 2008-04-03 10:40 +static int resize_bitmap(struct superblock *sb, struct allocspace *as, u64 newchunks, u64 new_basechunk, u64 new_metachunks) 2008-04-03 10:40 i searched the code for sb->copybuf 2008-04-03 10:43 flips: interjecting here...so you're killing static int update_bitmap... and replacing it with a resize_bitmap? 2008-04-03 10:44 just renaming a function 2008-04-03 10:44 to avoid confusion as above 2008-04-03 10:44 renames it- function remains the same? 2008-04-03 10:44 in c++, this function would be a method of a resize class 2008-04-03 10:44 function remains the same 2008-04-03 10:44 k 2008-04-03 10:45 willn: there were a couple reasons I abandoned LVM for test devices. 2008-04-03 10:46 1) LVM stacking was one of the bugs 2008-04-03 10:46 2) LVM had a 2T limit, and we wanted to test past that point. 2008-04-03 10:46 3) hard to create the LVM partition in the first place without already running virtualization (the UML test harness) 2008-04-03 11:02 I still don't get the chunk address calculations :( 2008-04-03 11:04 which part of the calculation? 2008-04-03 11:04 dld, what stacking problems does lvm have? 2008-04-03 11:06 Old bug, but the bio.throttle stuff hit lvm harder than raw disk. 2008-04-03 11:08 struct allocspace, u32 chunk_sectors_bits; /* Bits of number of sectors in a chunk. */ 2008-04-03 11:08 What does it exactly mean ? 2008-04-03 11:10 shift a 1 that many bits left and you get the chunk size 2008-04-03 11:10 hackers like to shift bits instead of multiply 2008-04-03 11:11 setup_sb(): sb->metadata.chunk_sectors_bits = bs_bits - SECTOR_BITS; 2008-04-03 11:11 bs_bits = 12 2008-04-03 11:11 SECTOR_BITS = 9 2008-04-03 11:11 chunk_sectors_bits = 3 ? 2008-04-03 11:12 bits -> shift 2008-04-03 11:12 should really be spelled "shift" 2008-04-03 11:12 oooh! 2008-04-03 11:13 Ok, while allocating a buffer. new_block() 2008-04-03 11:13 getblk(sb->metadev, newchunk << sb->metadata.chunk_sectors_bits, sb->metadata.allocsize); 2008-04-03 11:14 change from chunk addressing to sector addressing for getblk 2008-04-03 11:14 getblk should really use chunk addressing but it does not 2008-04-03 11:14 got to fix that one day, big job 2008-04-03 11:15 all the chunk addresses, are simply multiple of chunk_size. 2008-04-03 11:16 so, all I really need to do is.. multiply add offset (if necessary) ; and get my destination. 2008-04-03 11:30 -!- phoenix24(kglz@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-03 11:30 phoenix24, yes 2008-04-03 11:37 flips: 2^3-sectors in a chunk ? 2008-04-03 11:38 chunk size is some variable number of sectors 2008-04-03 11:38 sector = 2^9 2008-04-03 11:38 linux definition of sector 2008-04-03 11:39 or, ext3 default definition of sector ? 2008-04-03 11:39 4k 2008-04-03 11:40 no! 2008-04-03 11:40 ok, i get ur point 2008-04-03 11:41 unforunately, sector is defined as 512 in linux 2008-04-03 11:41 come from the old dos floppy disks 2008-04-03 11:42 linux defines "block" as 1K in some places, variable size in others 2008-04-03 11:42 goofy ;-) 2008-04-03 11:44 yea 2008-04-03 11:45 ok, lemme say it all again. 2008-04-03 11:46 for Zumastor, chunk size is variable nos of sectors. 2008-04-03 11:46 but the default chunk size turns to be 4k, ie of 8 sectors. 2008-04-03 11:46 DEFAULT_CHUNK_SIZE_BITS = 12 2008-04-03 11:46 the end of it! 2008-04-03 11:50 -!- erwan_taf(~erwan@81.80.43.67) has joined #zumastor 2008-04-03 12:22 zumastor-commits: [zumastor commit] r1522 - trunk/kernel/config 2008-04-03 12:30 -!- phoenix24_(vuehgf@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-03 14:34 phoenix24, the default chunk size is set as 4k in ddsnap, but zumastor sets the default chunksize as 16k. see /bin/zumastor:935 and /bin/zumastor:1003 2008-04-03 17:08 zumastor-commits: [zumastor commit] r1523 - in trunk: cbtb/host-scripts kernel/config 2008-04-03 19:36 -!- cbsmith(~xman@adsl-99-165-23-73.dsl.lsan03.sbcglobal.net) has joined #zumastor 2008-04-03 20:14 zumastor-commits: [PPA zumastor-team] Accepted: zumastor 0.8.0-r1523 (source) 2008-04-03 20:14 zumastor-commits: [PPA zumastor-team] Accepted: ddsnap 0.8.0-r1523 (source) 2008-04-03 20:14 zumastor-commits: [PPA zumastor-team] Accepted: zumastor 0.8.0-r1523m (source) 2008-04-03 20:14 zumastor-commits: [PPA zumastor-team] Accepted: ddsnap 0.8.0-r1523m (source) 2008-04-03 20:19 -!- tim_vimm(~Tim@cpe-76-90-128-140.socal.res.rr.com) has joined #zumastor 2008-04-03 21:01 -!- phoenix24(oqqtou@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-03 21:05 Hi 2008-04-03 21:05 What is the difference between, Squashing & Deleting ? 2008-04-03 21:08 good question ;) 2008-04-03 21:08 we realized they are basically the same, so we are removing squashing 2008-04-03 21:08 squashing is what happens when you auto-delete a snapshot which is still in use 2008-04-03 21:09 its a bogus concept that it is different from deleting 2008-04-03 21:21 ACTION waits for mixed test results to come back (i386, amd64, 3 different kernels) 2008-04-03 21:40 -!- tim_vimm(~Tim@cpe-76-90-128-140.socal.res.rr.com) has joined #zumastor 2008-04-03 22:01 -!- phoenix24_(qjh@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-03 23:04 -!- phoenix24(rayxg@h1119083.serverkompetenz.net) has joined #zumastor irc.oftc.net #zumastor log beginning Fri Apr 4 00:00:01 PDT 2008 2008-04-04 00:04 -!- phoenix24(hjc@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-04 01:10 -!- erwan_taf(~erwan@81.80.43.67) has joined #zumastor 2008-04-04 01:29 -!- erwan_taf(~erwan@81.80.43.67) has joined #zumastor 2008-04-04 01:31 -!- phoenix24(vah@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-04 03:05 -!- phoenix24(qrxtzt@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-04 06:00 -!- erwan_taf(~erwan@81.80.43.67) has joined #zumastor 2008-04-04 09:49 -!- tim_vimm(~Tim@cpe-76-90-128-140.socal.res.rr.com) has joined #zumastor 2008-04-04 10:09 good morning 2008-04-04 10:20 hi flips 2008-04-04 10:22 hi shapor 2008-04-04 10:37 ack 2008-04-04 10:37 my head hurts 2008-04-04 10:38 -!- pgquiles(~pgquiles@37.Red-83-44-237.dynamicIP.rima-tde.net) has joined #zumastor 2008-04-04 10:52 -!- phoenix24(tpm@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-04 10:57 willn, because? 2008-04-04 11:03 I spent alot of time debugging a test 2008-04-04 11:04 we were using two differnet ddsnap device names, and the tests dont clean up after themselves 2008-04-04 11:04 so my block devices were in use 2008-04-04 11:04 let's talk about how we will do the device exclusion for ddsnap 2008-04-04 11:05 willn, you know zumastor already does this at a higher level, kinda? 2008-04-04 11:05 I would be in favor of making the zumastor exclusion better, to start 2008-04-04 11:05 otherwise we will be tempted to do things like write flags into the ddsnap superblock 2008-04-04 11:06 a messy approach 2008-04-04 11:07 The test that was a problem was a non-zumastor test 2008-04-04 11:07 ddsnap_snapshot 2008-04-04 11:08 don't you want something like lvm has going on that says 'hey, theres already lvm metadata on this volume? is this sane?' 2008-04-04 11:11 except it's not about is there already metadata 2008-04-04 11:11 it's is there already a server running on that metadata 2008-04-04 11:11 very different logistics 2008-04-04 11:12 if you write stuff into the metadata saying "I'm a server and I own this" then how do you know if it was a crashed/gone server?) 2008-04-04 11:13 rather than writing into the metadata, a better way is to have a lockfile somewhere 2008-04-04 11:13 now... 2008-04-04 11:13 we can use the name of the metadata volume for a lock 2008-04-04 11:13 not really that great 2008-04-04 11:14 or the name of the socket, then if you lie about the socket that can be bypassed 2008-04-04 11:14 well 2008-04-04 11:14 ok 2008-04-04 11:14 we can try to make the block device a single-owner block device 2008-04-04 11:14 but it isn't 2008-04-04 11:15 because the server and N kernel clients open it 2008-04-04 11:15 no big rush in sorting this out by the way, since zumastor already should be taking care of not starting multiple servers 2008-04-04 11:15 well 2008-04-04 11:15 the agent should ensure we don't have multiple servers 2008-04-04 11:15 that is true 2008-04-04 11:16 and then zumastor should ensure we don't have multiple agents 2008-04-04 11:16 ok? 2008-04-04 11:16 how about usecounts on the underlying devices? 2008-04-04 11:17 where do we store the usecount? 2008-04-04 11:17 in the kernel? 2008-04-04 11:17 tell ya what, I will pencil in an lvm3 feature here 2008-04-04 11:17 doesn't the kernel already do this 2008-04-04 11:18 to prevent you from mounting a device twice? 2008-04-04 11:18 it has two ownership varieties: 1 or many 2008-04-04 11:18 we should set that bit 2008-04-04 11:18 can't 2008-04-04 11:18 why not 2008-04-04 11:18 our device is opened by N kernel clients + one server 2008-04-04 11:18 certainly dont want someone mounting the origin device underneath us 2008-04-04 11:19 will it cause the open to fail? 2008-04-04 11:19 yes 2008-04-04 11:19 well 2008-04-04 11:19 in kernel we dont have to worry about that 2008-04-04 11:19 we can teach it to be exclusive only for _userspace_ possibly 2008-04-04 11:19 as it now is, the kernel open will fail too 2008-04-04 11:19 yeah open on it should fail 2008-04-04 11:20 in userspace 2008-04-04 11:20 and we need to teach the userspace opener not to count kernel opens 2008-04-04 11:20 ok, I will look at that 2008-04-04 11:21 will be another patch against core blockdev 2008-04-04 11:21 another step on the way to lvm3 2008-04-04 11:23 "userspace-only exclusive" <- lvm3 feature 2008-04-04 11:41 -!- pgquiles(~pgquiles@172.Red-83-40-81.dynamicIP.rima-tde.net) has joined #zumastor 2008-04-04 11:48 -!- pgquiles(~pgquiles@219.Red-88-25-135.staticIP.rima-tde.net) has joined #zumastor 2008-04-04 11:51 how does this sound as a title for the linuxtag paper? "Work in progress: LVM3 Advanced Volume Manager for Linux" 2008-04-04 11:52 or "LVM3 Advanced Volume Manager for Linux: Requirements, Design and Work in Progress" 2008-04-04 12:00 -!- phoenix24(oxxr@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-04 12:03 -!- pgquiles(~pgquiles@79.144.192.176) has joined #zumastor 2008-04-04 14:02 -!- pgquiles(~pgquiles@103.Red-88-23-190.staticIP.rima-tde.net) has joined #zumastor 2008-04-04 15:19 After a period of rapid advancement with the introduction of Device Mapper in Linux 2.4 and a redesigned block layer in Linux 2.6 there followed some years of relative stasis in storage systems development during which other operating systems such as Sun Solaris and FreeBSD did not stand still. Linux now has some catching up to do. LVM3 is a new project founded by Daniel Phillips of Google and others, to incorporate the best ideas of Device Mapper into the 2008-04-04 15:19 generic Linux block layer so that all block devices on Linux may take advantage of modern volume management techniques such as mirroring, snapshotting, replication, encryption, stacking, physical migration while in use, and many other possibilities. LVM3 will introduce the ability to boot directly from a virtual device without userspace support, and hence without needing an initrd. In the process of implementing LVM3, the generic block layer will be clean 2008-04-04 15:19 ed up to make it more efficient and less prone to deadlock. This paper introduces the key design concepts at the both kernel and userspace level and includes an update on work in progress. 2008-04-04 15:19 -- comments? 2008-04-04 15:20 sounds good 2008-04-04 15:20 forgot resizing ;) 2008-04-04 15:29 -!- pgquiles(~pgquiles@103.Red-88-23-190.staticIP.rima-tde.net) has joined #zumastor 2008-04-04 16:22 Hooray 2008-04-04 16:22 I think the multi node tests work 2008-04-04 17:22 -!- metzZzz(~metz@p5090D241.dip.t-dialin.net) has joined #zumastor 2008-04-04 17:23 -!- metzZzz(~metz@p5090D241.dip.t-dialin.net) has left #zumastor 2008-04-04 19:13 -!- DavidSev(david@lolcat.mercenariesguild.net) has joined #zumastor 2008-04-04 23:08 -!- tim_vimm(~Tim@cpe-76-90-128-140.socal.res.rr.com) has joined #zumastor 2008-04-04 23:15 -!- phoenix24(srixgh@h1119083.serverkompetenz.net) has joined #zumastor irc.oftc.net #zumastor log beginning Sat Apr 5 00:00:01 PDT 2008 2008-04-05 00:47 -!- phoenix24(hoaarwj@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-05 01:48 -!- phoenix24_(vksq@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-05 01:48 this silence is killing me! 2008-04-05 02:02 time to sleep here 2008-04-05 02:02 feel free to carry on a monologue 2008-04-05 02:03 I often do 2008-04-05 02:10 -!- pgquiles(~pgquiles@102.Red-88-0-159.dynamicIP.rima-tde.net) has joined #zumastor 2008-04-05 02:48 -!- phoenix24(ukz@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-05 03:48 -!- phoenix24_(nbrh@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-05 08:09 -!- pgquiles(~pgquiles@1.Red-83-40-80.dynamicIP.rima-tde.net) has joined #zumastor 2008-04-05 08:12 -!- phoenix24(mvkyjw@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-05 08:13 What methods can be used to optimize the seektimes in a BTree ? 2008-04-05 08:14 Such that lookup_time(data_block) in a FS, in the FS can be minimized. 2008-04-05 08:15 All I did was take a good B+Tree, well.. that is not enough :( 2008-04-05 08:26 Isn't there any restriction Linux kernel imposes on how much memory, One program be allocated ? 2008-04-05 08:26 Or how can I do it. 2008-04-05 09:04 -!- charlesnw(~charles@cpe-75-84-67-98.socal.res.rr.com) has joined #zumastor 2008-04-05 09:05 -!- charlesnw(~charles@cpe-75-84-67-98.socal.res.rr.com) has left #zumastor 2008-04-05 10:59 -!- phoenix24(wqyjz@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-05 11:08 -!- pgquiles(~pgquiles@164.Red-83-41-45.dynamicIP.rima-tde.net) has joined #zumastor 2008-04-05 11:53 phoenix24, the main method of optimizing seek times in a btree is to place leafs that are logically contiguous physically near each other 2008-04-05 11:54 phoenix24, there are ulimits to enforce per-process memory allocation limits, but the default is not to use them 2008-04-05 11:55 phoenix24, seek time in a btree can also be reduced by storing the leaf data more compactly so that fewer leaves need to be loaded 2008-04-05 12:22 -!- pgquiles(~pgquiles@164.Red-83-41-45.dynamicIP.rima-tde.net) has joined #zumastor 2008-04-05 12:23 http://us.shuttle.com/kpc/techspecs.html <- starts at $229 w/ dual processors 2008-04-05 12:24 linux preinstalled 2008-04-05 12:25 I'm on the verge of ordering one complete with wireless, dvd writer, backpack carrying case, 2 GB dram, 260 GB disk for $509 2008-04-05 12:26 100 wall power supply, should be acceptably quiet 2008-04-05 12:26 not as quiet as my pentium-m box with external power supply I expect 2008-04-05 12:26 or the fit pc running on 5 watts 2008-04-05 13:55 -!- phoenix24(bst@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-05 13:55 flips: ping 2008-04-05 13:55 I'm sorry, I wasn't at my desk. 2008-04-05 13:58 can't I also, max-number of elements in a leaf node (increasing leaf capacity).. thereby effectively decreasing the Btree height. 2008-04-05 14:00 btw, anyone using MAC OS X Leopard ?.. specially its TIME-MACHINE application. 2008-04-05 14:14 Apple's Time-Machine : http://www.apple.com/macosx/features/timemachine.html 2008-04-05 14:21 -!- tim_vimm(~Tim@cpe-76-90-128-140.socal.res.rr.com) has joined #zumastor 2008-04-05 14:25 -!- tim_vimm(~Tim@cpe-76-90-128-140.socal.res.rr.com) has joined #zumastor 2008-04-05 14:35 -!- tim_vimm(~Tim@cpe-76-90-128-140.socal.res.rr.com) has joined #zumastor 2008-04-05 14:55 -!- phoenix24_(njnva@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-05 15:56 -!- phoenix24(ybtx@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-05 21:15 -!- phoenix24(knuiy@h1119083.serverkompetenz.net) has joined #zumastor irc.oftc.net #zumastor log beginning Sun Apr 6 00:00:01 PDT 2008 2008-04-06 02:29 -!- phoenix24(hkooi@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-06 02:35 I've been reading on "Tiger Shark Multimedia FS", was designed by IBM for high availability, high scalability On demand video server. 2008-04-06 02:36 Its got lot of inbuilt fail-safes and faul-tolerant by design, leaving room for for no "single point of failure". 2008-04-06 02:37 Both the Zumastor client & the server.. can replace each other once the backup is complete, and in a consistent state. 2008-04-06 02:38 but, Zumastor still leaves room for a single point of failure ? 2008-04-06 02:43 does Zumastor support "striping", somebody has to update me here. 2008-04-06 03:11 Are the kernel patches in Zumastor against disk I/O scheduling ? 2008-04-06 03:29 -!- phoenix24_(tdwkwpa@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-06 04:45 -!- phoenix24(zrz@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-06 07:31 -!- phoenix24(igrmepe@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-06 07:32 When joining a new project. . what are the good ways to catchup with the existing code-implementation ? 2008-04-06 09:42 -!- phoenix24(jrar@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-06 11:02 -!- phoenix24(gvz@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-06 12:03 -!- phoenix24_(hpbwjrn@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-06 13:03 -!- phoenix24(ddvyiv@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-06 13:47 -!- pgquiles(~pgquiles@208.Red-88-16-32.dynamicIP.rima-tde.net) has joined #zumastor 2008-04-06 14:03 -!- pgquiles(~pgquiles@208.Red-88-16-32.dynamicIP.rima-tde.net) has joined #zumastor 2008-04-06 14:03 -!- phoenix24_(rbqprts@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-06 15:04 -!- phoenix24(untw@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-06 16:04 -!- phoenix24(glhz@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-06 21:18 -!- cbsmith(~xman@adsl-99-165-23-73.dsl.lsan03.sbcglobal.net) has joined #zumastor 2008-04-06 22:03 -!- phoenix24(rfka@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-06 22:18 Hi 2008-04-06 22:18 flips: Is the LVM3 hosted somewhere ? 2008-04-06 22:19 phoenix24, the patches are on zumastor.org 2008-04-06 22:19 will set up something better soon 2008-04-06 22:21 Are the kernel patches in Zumastor against disk I/O scheduling ? 2008-04-06 23:03 -!- phoenix24_(rbv@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-06 23:07 flips: ping, can you spare me few mins ? irc.oftc.net #zumastor log beginning Mon Apr 7 00:00:01 PDT 2008 2008-04-07 00:03 -!- phoenix24(yawrj@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-07 01:22 -!- erwan_taf(~erwan@81.80.43.67) has joined #zumastor 2008-04-07 02:32 -!- erwan_taf(~erwan@81.80.43.67) has joined #zumastor 2008-04-07 03:54 -!- phoenix24(yulz@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-07 06:27 -!- MaZe(~MaZe@193.120.148.177) has joined #zumastor 2008-04-07 09:38 -!- phoenix24(ibbrh@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-07 10:18 -!- mitchg(~mitchg@207.47.98.129.static.nextweb.net) has joined #zumastor 2008-04-07 10:40 good morning 2008-04-07 10:49 Howdy 2008-04-07 10:59 -!- phoenix24(qog@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-07 11:20 scramjet time 2008-04-07 11:59 -!- phoenix24_(bivbqe@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-07 12:15 heh! funny! 2008-04-07 12:59 -!- phoenix24_(avs@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-07 13:02 :P http://xkcd.com/403/ 2008-04-07 13:43 haha 2008-04-07 14:00 -!- phoenix24(nyv@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-07 14:22 -!- dkegel(~chatzilla@207.47.98.129.static.nextweb.net) has joined #zumastor 2008-04-07 14:23 xman? 2008-04-07 15:41 Oy 2008-04-07 15:41 our kernels have broken version numbers 2008-04-07 16:39 -!- pgquiles(~pgquiles@208.Red-88-16-32.dynamicIP.rima-tde.net) has joined #zumastor 2008-04-07 16:45 ACTION bookmarks xkcd 2008-04-07 16:46 Looks like i've still got 3 bugs to fix 2008-04-07 16:47 fix one and another will rise up to take its place 2008-04-07 16:47 there always were three bugs and there always will be three bugs, for ever and ever amen 2008-04-07 16:47 on unlucky projects, ten pop up for every three you fix... 2008-04-07 16:48 i'm poking the bigcopy test, so i can fix #74 2008-04-07 16:50 -!- pgquiles(~pgquiles@208.Red-88-16-32.dynamicIP.rima-tde.net) has joined #zumastor 2008-04-07 16:51 though, my head is killing me 2008-04-07 16:56 -!- pgquiles_(~pgquiles@208.Red-88-16-32.dynamicIP.rima-tde.net) has joined #zumastor 2008-04-07 17:00 -!- pgquiles__(~pgquiles@208.Red-88-16-32.dynamicIP.rima-tde.net) has joined #zumastor 2008-04-07 17:37 Fixed the doc bug 2008-04-07 17:37 I reported the kernel version bug 2008-04-07 17:38 and we've got cbtb bashisms that are an issue 2008-04-07 17:38 I need to clean up tests so they use sane names, so we can have a formula for writing new tests (eg, zumatest or testvol for volume naming) 2008-04-07 17:47 -!- pgquiles__(~pgquiles@208.Red-88-16-32.dynamicIP.rima-tde.net) has joined #zumastor 2008-04-07 18:26 Created issue 103 for cbtb cleanup 2008-04-07 18:26 hi pg 2008-04-07 18:26 and dkegel flips: see issue 102 for the bloody slow bit 2008-04-07 22:02 -!- phoenix24(qvtddxh@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-07 22:15 like the commits.. can we also have the Issue's displayed at IRC ? 2008-04-07 23:27 -!- phoenix24(gzrkc@h1119083.serverkompetenz.net) has joined #zumastor irc.oftc.net #zumastor log beginning Tue Apr 8 00:00:01 PDT 2008 2008-04-08 00:27 -!- phoenix24_(kzfn@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-08 01:28 -!- phoenix24(bgjse@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-08 02:28 -!- phoenix24_(rnwhge@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-08 02:58 -!- MaZe(~MaZe@89.100.181.155) has joined #zumastor 2008-04-08 03:28 -!- phoenix24(owjibv@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-08 04:28 -!- phoenix24_(zruvotc@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-08 04:36 the, dm-ddsnap defines the kernel module. 2008-04-08 04:52 Which package need to be installed for Man pages on Kernel functions ? 2008-04-08 04:57 -!- MaZe(~MaZe@193.120.148.177) has joined #zumastor 2008-04-08 05:00 on debian its (I think) linux-doc-* 2008-04-08 05:28 -!- phoenix24(szav@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-08 06:29 -!- phoenix24_(axqodbr@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-08 07:23 -!- MaZe(~MaZe@193.120.148.177) has joined #zumastor 2008-04-08 08:01 -!- phoenix24(okd@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-08 09:01 -!- MaZe(~MaZe@193.120.148.177) has joined #zumastor 2008-04-08 10:32 -!- phoenix24(aaeczdc@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-08 10:41 phoenix24: Unfortunately there is no rss/atom feed from code.google.com at the moment 2008-04-08 10:43 ah! yes 2008-04-08 10:43 I'll rig something, sec 2008-04-08 10:45 willn-colo: rss-add zissues http://pipes.yahoo.com/pipes/pipe.run?_id=YNkTgZMF3RG3JMXbjtzu1g&_render=rss 2008-04-08 10:45 rss item added 2008-04-08 10:45 willn-colo: rss-watch zissues 300 2008-04-08 10:45 watcher thread started 2008-04-08 10:48 pipes is pretty darn nifty 2008-04-08 10:48 Yeh, I just checked the RSS feed you just created. 2008-04-08 10:53 willn: howd you do that? 2008-04-08 10:58 creating pipes ? 2008-04-08 11:00 http://pipes.yahoo.com/pipes/ -> has all info you'll need. 2008-04-08 11:00 kewl 2008-04-08 11:01 shapor: http://pipes.yahoo.com/pipes/pipe.info?_id=YNkTgZMF3RG3JMXbjtzu1g may have more isnight 2008-04-08 11:01 I don't know if it will let you see the 'source' 2008-04-08 11:02 If anyone wants to chime in on the billion patches i've been adding to the issue tracker, I won't complain ;) 2008-04-08 11:03 It does allow to you view the source : http://pipes.yahoo.com/pipes/pipe.edit?_id=YNkTgZMF3RG3JMXbjtzu1g 2008-04-08 11:03 ok if I just applaud? 2008-04-08 11:03 phoenix24: Yea, I wasn't sure if I had to click the 'make it public' button or not 2008-04-08 11:03 Hi dan! 2008-04-08 11:04 hi phoenix! 2008-04-08 11:04 xman here? 2008-04-08 11:05 zissues: Issue 105 in zumastor: missing zumastor forget schedule 2008-04-08 11:06 cool! 2008-04-08 11:42 hi xman 2008-04-08 11:42 hi dank 2008-04-08 11:44 -!- phoenix24(lojmxrn@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-08 11:53 dkegel: Updated issue 83 for your perusal 2008-04-08 11:55 zissues: Issue 77 in zumastor: 2.6.24.2-i386: bootup stops 2008-04-08 12:05 flips: Have you taken any action on 99? 2008-04-08 12:11 zissues: Issue 83 in zumastor: Add XFS quirks to zumastor-howto 2008-04-08 12:12 hi xman 2008-04-08 12:12 xman, how is your aio work? 2008-04-08 12:34 flips: see 82 2008-04-08 12:44 -!- phoenix24_(dnnpdxs@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-08 12:46 zissues: Issue 82 in zumastor: -h option is overloaded for zumastor define volume 2008-04-08 13:16 jiayingz: coming along 2008-04-08 13:16 jiayingz: using eventfd 2008-04-08 13:17 -!- jiayingz_(~jiayingz@207.47.98.129.static.nextweb.net) has joined #zumastor 2008-04-08 13:19 cbsmith, the crash problem has been solved with eventfd? 2008-04-08 13:20 jiayingz: crash problem? There was a hang problem that seems to not be happening with it. 2008-04-08 13:21 cbsmith, could you post the new patch to the list and the instructions on how to try it? 2008-04-08 13:22 jiayingz: Yeah, working on getting that together right now. 2008-04-08 13:22 jiayingz: But it doesn't fix any crash problem that I was aware of. 2008-04-08 13:22 cbsmith, my mistake. i meaned hanging problem 2008-04-08 13:22 jiayingz: ah, okay. 2008-04-08 13:23 jiayingz: then you should see something in the next couple of hours. 2008-04-08 13:23 cbsmith, when u post, could u send the patch as attachment? it is easier to apply the patch in that way 2008-04-08 13:24 jiayingz: no problemo 2008-04-08 13:31 zissues: Issue 76 in zumastor: 2.6.24.2-um: mkfs.xfs halts 2008-04-08 13:44 -!- jiayingz__(~jiayingz@207.47.98.129.static.nextweb.net) has joined #zumastor 2008-04-08 14:04 -!- tim_vimm(~Tim@cpe-76-90-128-140.socal.res.rr.com) has joined #zumastor 2008-04-08 14:28 -!- jiayingz_(~jiayingz@207.47.98.129.static.nextweb.net) has joined #zumastor 2008-04-08 14:35 flips: ping 2008-04-08 14:35 willn, pong 2008-04-08 14:38 willn, I see the problem with 82 ;) 2008-04-08 14:38 willn, 99 is low priority because zumastor should not start more than one server 2008-04-08 14:39 this then becomes a feature request for the agent component ddsnap, without an obvious right way to proceed 2008-04-08 14:44 ok 2008-04-08 14:57 zumastor-commits: [zumastor commit] r1526 - in trunk/cbtb/tests: 1 2 2008-04-08 14:57 -!- tim_vimm(~Tim@cpe-76-90-128-140.socal.res.rr.com) has joined #zumastor 2008-04-08 14:58 google groups: messages delayed by a "long time" (tm) 2008-04-08 15:05 -!- jiayingz__(~jiayingz@207.47.98.129.static.nextweb.net) has joined #zumastor 2008-04-08 15:07 zumastor-commits: [zumastor commit] r1524 - branches/0.7/doc 2008-04-08 15:11 zissues: Issue 46 in zumastor: Ease zumastor upgrade when superblock structure changes 2008-04-08 15:13 zumastor-commits: [zumastor commit] r1525 - trunk/ddsnap 2008-04-08 15:17 -!- tim_vimm(~Tim@cpe-76-90-128-140.socal.res.rr.com) has joined #zumastor 2008-04-08 15:18 zumastor-commits: [zumastor commit] r1527 - trunk/doc 2008-04-08 15:20 -!- zbot(wan@209.67.252.126) has joined #zumastor 2008-04-08 15:21 zbot: rss-watch zcommits 300 2008-04-08 15:21 watcher thread started 2008-04-08 15:35 zissues: Issue 96 in zumastor: mkfs.xfs hanging in 2.6.24 trunk UML tests 2008-04-08 16:01 zcommits: [zumastor commit] r1529 - trunk/cbtb/host-setup 2008-04-08 16:05 -!- jiayingz_(~jiayingz@207.47.98.129.static.nextweb.net) has joined #zumastor 2008-04-08 16:06 zcommits: [zumastor commit] r1530 - trunk/cbtb/host-setup 2008-04-08 17:16 -!- pgquiles__(~pgquiles@208.Red-88-16-32.dynamicIP.rima-tde.net) has joined #zumastor 2008-04-08 17:46 zcommits: [zumastor commit] r1531 - trunk/cbtb/host-scripts 2008-04-08 18:12 -!- MaZ1(~MaZe@89.100.181.155) has joined #zumastor 2008-04-08 18:12 -!- phoenix24_(yolihbb@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-08 19:58 -!- phoenix24(yovxsf@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-08 20:52 -!- jiayingz_(~jiayingz@207.47.98.129.static.nextweb.net) has joined #zumastor 2008-04-08 21:50 -!- phoenix24(maz@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-08 22:51 -!- phoenix24(tuki@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-08 23:07 -!- jiayingz_(~jiayingz@cpe-76-167-221-105.socal.res.rr.com) has joined #zumastor irc.oftc.net #zumastor log beginning Wed Apr 9 00:00:01 PDT 2008 2008-04-09 00:04 -!- phoenix24(wsnjya@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-09 00:52 happy hour : I just wrote my first linux device driver.. learning from zumastor kernel module 2008-04-09 01:04 -!- phoenix24_(rrmgfse@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-09 02:04 -!- phoenix24_(lubrc@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-09 02:15 -!- pgquiles_(~pgquiles@14.Red-79-144-193.staticIP.rima-tde.net) has joined #zumastor 2008-04-09 02:46 Does /dev/kmem exist on the Vanilla kernel, or its patched by my Ubuntu Kernel ? 2008-04-09 02:47 I seem to get an impression that /dev/kmem has be depricated. => http://lwn.net/Articles/147901/ 2008-04-09 03:05 -!- phoenix24(dlsb@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-09 03:22 -!- MaZ1(~MaZe@193.120.148.177) has joined #zumastor 2008-04-09 04:04 -!- phoenix24_(vhnqjvr@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-09 05:16 -!- MaZ1(~MaZe@193.120.148.177) has joined #zumastor 2008-04-09 05:21 -!- phoenix24(tsnps@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-09 07:09 -!- MaZ1(~MaZe@193.120.148.177) has joined #zumastor 2008-04-09 07:47 -!- tim_vimm(~Tim@cpe-76-90-128-140.socal.res.rr.com) has joined #zumastor 2008-04-09 07:54 ACTION reminds everyone hes OOO until this afternoon 2008-04-09 09:54 ddt-zumastor: Change change_bits from start/finish to start/count from 2008-04-09 09:54 ddt-zumastor: update_bitmap -> adjust_bitmap; function name should not imply it is used in 2008-04-09 09:54 ddt-zumastor: Must flush deferred allocs on any clean save to avoid losing allocs due to no 2008-04-09 10:07 zissues: Issue 5 in zumastor: btree RAM usage is excessive 2008-04-09 10:22 zissues: Issue 106 in zumastor: Reduce the overhead of btree updates during origin writes 2008-04-09 10:22 zissues: Issue 52 in zumastor: Reduce the overhead of bitmap updates during origin writes 2008-04-09 10:27 zissues: Issue 108 in zumastor: implement and test write-anywhere model 2008-04-09 10:27 zissues: Issue 107 in zumastor: Test Zumastor performance with nvram 2008-04-09 10:38 -!- dkegel(~chatzilla@207.47.98.129.static.nextweb.net) has joined #zumastor 2008-04-09 10:52 zissues: Issue 109 in zumastor: Automatic performance testing framework 2008-04-09 10:58 -!- cbsmith(~xman@adsl-99-165-23-73.dsl.lsan03.sbcglobal.net) has joined #zumastor 2008-04-09 11:33 -!- phoenix24(vmxazfc@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-09 11:34 -!- jiayingz_(~jiayingz@207.47.98.129.static.nextweb.net) has joined #zumastor 2008-04-09 11:58 Hi everyone! 2008-04-09 12:15 !zbot 2008-04-09 12:16 hi phoenix24 2008-04-09 12:17 Hi jiayingz 2008-04-09 12:17 i think kmem is disabled by default now 2008-04-09 12:18 ok, I guess Ubuntu Kernel team adds it then. 2008-04-09 12:19 What does the dm-ddsnap Kernel module accomplish ? 2008-04-09 12:20 it is the kernel part of code of ddsnap 2008-04-09 12:20 get bio request from device mapper layer and does special ddsnap handling 2008-04-09 12:20 I've never done device drivers .. so couldn't really follow that part 2008-04-09 12:20 ok 2008-04-09 12:22 basically, the kernel dm-ddsnap code look at each bio request. if it is origin write, it sends a request to the user-level ddsnap server which may perform copyout to avoid overwriting old versions of data 2008-04-09 12:22 origin reads can just go through 2008-04-09 12:22 okPF_LESS_THROTTLE 2008-04-09 12:22 for snapshot reads, the kernel sends a request to user-level ddsnad which will put a lock to the read block 2008-04-09 12:23 PF_LESS_THROTTLE patch? 2008-04-09 12:23 I was about to ask.. searching google now. 2008-04-09 12:23 it gives a process certain previlage to get more memory request 2008-04-09 12:23 ok 2008-04-09 12:24 because ddsnap handles io request. its memory request needs to be satisfied with high priority 2008-04-09 12:25 the purpose of PF_LESS_THROTTLE and PF_MEMLOCK is to avoid deadlock during the handling of io requests 2008-04-09 12:26 http://lkml.org/lkml/2007/12/5/346 2008-04-09 12:26 kmem_cache_create() is used to create a kernel cache 2008-04-09 12:26 yes 2008-04-09 12:26 and, we set flags disabling swapout. 2008-04-09 12:27 in ddsnapd? yes 2008-04-09 12:27 -!- phoenix24_(ovanot@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-09 12:27 what does dm_register_target() do ? 2008-04-09 12:28 register as a device mapper target 2008-04-09 12:29 the kernel part of code is quite complex. the good thing is you don't need to fully understand that part to develop ddsnap userland code 2008-04-09 12:29 there is a design note on zumastor.org 2008-04-09 12:29 http://zumastor.googlecode.com/svn/trunk/doc/desimplnotes.html 2008-04-09 12:30 it explains some basic process on how ddsnap handles io requests 2008-04-09 12:31 and etc. 2008-04-09 13:25 Thank you jiayingz.. the design notes is very helpful :) 2008-04-09 14:14 zcommits: [zumastor commit] r1532 - in trunk: ddsnap/man zumastor/man 2008-04-09 15:13 -!- MaZ1(~MaZe@89.100.181.155) has joined #zumastor 2008-04-09 15:29 -!- natalie(~nataliep@207.47.98.129.static.nextweb.net) has joined #zumastor 2008-04-09 15:47 -!- tim_vimm_(~Tim@cpe-76-90-128-140.socal.res.rr.com) has joined #zumastor 2008-04-09 15:50 ACTION blinks 2008-04-09 16:01 -!- pgquiles__(~pgquiles@206.Red-88-25-134.staticIP.rima-tde.net) has joined #zumastor 2008-04-09 16:04 -!- pgquiles_(~pgquiles@210.Red-88-17-196.staticIP.rima-tde.net) has joined #zumastor 2008-04-09 19:22 -!- tim_vimm(~Tim@cpe-76-90-128-140.socal.res.rr.com) has joined #zumastor 2008-04-09 20:05 -!- flipz(~phlipz@adsl-63-202-13-187.dsl.snfc21.pacbell.net) has joined #zumastor 2008-04-09 20:38 -!- phoenix24(ozr@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-09 21:38 -!- phoenix24_(zxju@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-09 22:03 -!- jiayingz_(~jiayingz@cpe-76-167-221-105.socal.res.rr.com) has joined #zumastor 2008-04-09 22:51 -!- pgquiles_(~pgquiles@25.Red-88-16-37.dynamicIP.rima-tde.net) has joined #zumastor 2008-04-09 23:47 -!- phoenix24(yalvfuf@h1119083.serverkompetenz.net) has joined #zumastor irc.oftc.net #zumastor log beginning Thu Apr 10 00:00:01 PDT 2008 2008-04-10 00:52 -!- phoenix24(fmofxil@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-10 01:53 -!- phoenix24(cryb@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-10 03:12 -!- phoenix24_(ynzk@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-10 03:41 -!- erwan_taf(~erwan@81.80.43.67) has joined #zumastor 2008-04-10 04:42 good morning 2008-04-10 05:08 -!- phoenix24_(gagasn@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-10 05:09 Hi 2008-04-10 05:14 -!- MaZ1(~MaZe@193.120.148.177) has joined #zumastor 2008-04-10 07:14 -!- tim_vimm(~Tim@cpe-76-90-128-140.socal.res.rr.com) has joined #zumastor 2008-04-10 07:14 morning tim_vimm 2008-04-10 07:14 good morning traveling flipz 2008-04-10 07:15 nap time here 2008-04-10 07:15 been working late 2008-04-10 07:15 been coding all night? 2008-04-10 07:15 conferring with europeans 2008-04-10 07:15 and coding 2008-04-10 07:15 those darn euros 2008-04-10 07:15 really 2008-04-10 07:15 should move to a decent time zone 2008-04-10 07:16 gimme a sec, and I'll appear in another place 2008-04-10 07:17 ok, got my coffee 2008-04-10 07:31 -!- tim_vimm(~Tim@cpe-76-90-128-140.socal.res.rr.com) has left #zumastor 2008-04-10 08:11 flipz: are you in Europe? 2008-04-10 09:34 pgquiles_, I am in mountain view california 2008-04-10 09:37 flipz: oh, that "[16:15] conferring with europeans" made me think you were in EU :-) 2008-04-10 09:37 up in the middle of the night talking to them 2008-04-10 09:37 much as you do from time to time 2008-04-10 09:37 :-) 2008-04-10 09:41 -!- tim_vimm(~Tim@cpe-76-90-128-140.socal.res.rr.com) has joined #zumastor 2008-04-10 10:00 -!- jiayingz_(~jiayingz@207.47.98.129.static.nextweb.net) has joined #zumastor 2008-04-10 10:06 -!- MaZ1(~MaZe@193.120.148.177) has joined #zumastor 2008-04-10 10:08 -!- MaZ1(~MaZe@193.120.148.177) has joined #zumastor 2008-04-10 10:09 -!- MaZ1(~MaZe@193.120.148.177) has joined #zumastor 2008-04-10 10:16 -!- MaZ1(~MaZe@193.120.148.177) has joined #zumastor 2008-04-10 10:19 -!- MaZe(~MaZe@193.120.148.177) has joined #zumastor 2008-04-10 10:20 -!- MaZe(~MaZe@193.120.148.177) has joined #zumastor 2008-04-10 10:25 -!- MaZe(~MaZe@193.120.148.177) has joined #zumastor 2008-04-10 10:46 -!- mitchg(~mitchg@207.47.98.129.static.nextweb.net) has joined #zumastor 2008-04-10 10:55 -!- cbsmith(~xman@adsl-99-165-23-73.dsl.lsan03.sbcglobal.net) has joined #zumastor 2008-04-10 10:57 -!- jiayingz_(~jiayingz@207.47.98.129.static.nextweb.net) has joined #zumastor 2008-04-10 11:05 -!- phoenix24(jsif@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-10 11:25 -!- zumalog(~zumalog@yzf.shapor.com) has joined #zumastor 2008-04-10 12:07 -!- phoenix24(tpfetp@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-10 12:10 -!- tim_vimm(~Tim@cpe-76-90-128-140.socal.res.rr.com) has joined #zumastor 2008-04-10 13:07 -!- phoenix24_(ysdeds@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-10 13:36 -!- flipz(~phlipz@216.239.45.19) has joined #zumastor 2008-04-10 13:37 good afternoon 2008-04-10 13:45 hi from socal :) 2008-04-10 13:57 I'm looking for a cheap RC car, but I don't think our ratshack has any 2008-04-10 13:57 Shops these days.. 2008-04-10 13:58 willn how cheap? 2008-04-10 13:59 zip zap cheap? 2008-04-10 14:00 I'm looking for ~10"x6" or so, under 50 2008-04-10 14:01 In a perfect world, something with servos for front steering. Unlikely to find that at that price 2008-04-10 14:07 ACTION loads up the thumper again 2008-04-10 14:07 Nice to have working amd64 packages this time aroudn 2008-04-10 14:31 -!- phoenix24(oej@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-10 14:33 Hi 2008-04-10 14:56 -!- dkegel(~chatzilla@207.47.98.129.static.nextweb.net) has joined #zumastor 2008-04-10 14:56 flips / flipz, ping? 2008-04-10 15:10 ACTION maybe finds an amd64 bug... confirming 2008-04-10 15:12 willn: ? 2008-04-10 15:12 Interesting behavior replicating from i386 -> amd64 2008-04-10 15:13 by interesting I mean 2008-04-10 15:13 willn: that's a 32-bit bug then. ;-) 2008-04-10 15:13 Thu Apr 10 18:13:29 2008: [30837] error_message_handler: downstream server reason why send delta failed : incomplete SEND_DELTA request sent by client: length 28, size 32 2008-04-10 15:16 dammit, is how can I do memory barriers/CAS in userspace in a semi-portable way with Linux? 2008-04-10 15:16 Oh, and using only Zumastor's C of course. 2008-04-10 15:16 ACTION grumbles about the virtues of C++ 2008-04-10 15:16 ah, found it 2008-04-10 15:20 looks like a real bug. Just rebuild the amd64 packages by hand with the same issue 2008-04-10 15:20 willn: probably a bug 2008-04-10 15:23 zcommits: [zumastor commit] r1533 - in trunk: ddsnap ddsnap/man zumastor/bin zumastor/man 2008-04-10 15:24 Thu Apr 10 18:23:53 2008: [1872] event_parse_options: invalid count in DDSNAP_COUNT 2008-04-10 15:24 that is a fake error 2008-04-10 15:24 heh 2008-04-10 15:25 I'm getting lost in fake errors ^-^ 2008-04-10 15:25 feel free to clean them :) 2008-04-10 15:29 Will, did you really find a 32/64 interaction bug? 2008-04-10 15:30 dkegel: Well, it isn't working. 2008-04-10 15:30 Cool, you found it before any user did :-) 2008-04-10 15:32 I'm curious why 2008-04-10 15:33 also, awsome spelling mistake in 'replication' on line 1729 of ddsnap.c 2008-04-10 15:34 but since I mispelled awesome, it is all par for the course 2008-04-10 15:34 http://www.kegel.com/kerspell/ 2008-04-10 15:34 http://www.kegel.com/wine/spellingpolice/ 2008-04-10 15:38 I wonder if its the thumper being silly... 2008-04-10 15:39 -!- flipz(~phlipz@216.239.45.19) has joined #zumastor 2008-04-10 15:40 willn, could you post the test to the list? I can try it on my 64bit testing machine. 2008-04-10 15:41 i'm going to try 64->64 and non thumper 32 -> 64 2008-04-10 15:41 it is possible to be a 32/64 bug. we haven't tested that a lot 2008-04-10 15:41 I really just defined a 10G origin 20G snap and tried replication 2008-04-10 15:42 -!- phoenix24(jnkrt@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-10 15:51 So, I think we have our first documented evidence of a unit test finding a bug. Of course, it was in my code. ;-) 2008-04-10 15:54 Hey, that counts. 2008-04-10 15:54 In fact, the best time for tests to find bugs is before the first commit, so you got it exactly right. 2008-04-10 15:59 i also used ur unit test 2008-04-10 16:00 i guess we can close issue 29 2008-04-10 16:02 hmm 2008-04-10 16:02 amd64 -> amd64 worked fine 2008-04-10 16:02 granted, using loop devices 2008-04-10 16:04 Hey Jiayingz, I just noticed those synchronization primitives I used are relatively new. 2008-04-10 16:04 So the compile may not work. 2008-04-10 16:06 What means by a 32/64 interaction bug? 2008-04-10 16:06 replication not working between a 32 bit host and a 64 bit host 2008-04-10 16:06 ok 2008-04-10 16:09 still broken 32->64 2008-04-10 16:09 hmm 2008-04-10 16:09 strange. 2008-04-10 16:15 testpaio.o: In function `aio_event_reader':../paio.c:16: undefined reference to `__sync_bool_compare_and_swap' 2008-04-10 16:17 chris, i think we need -lpthread in tests/Makefile 2008-04-10 16:17 filed bug 110 re: 32->64 2008-04-10 16:29 zissues: Issue 110 in zumastor: 64/32bit interaction issues 2008-04-10 16:31 -pthread is preferred to -lpthread, it also does a needed #define 2008-04-10 16:34 willn, i also saw the 32->64 problem on my testing machine 2008-04-10 16:35 dkegel is right, I thought I did that, but I guess I failed to. 2008-04-10 16:35 jiayingz: Do you have gcc-4.1 handy? 2008-04-10 16:36 cbsmith, no 2008-04-10 16:37 cbsmith, is there any generial api we can use instead of `__sync_bool_compare_and_swap' 2008-04-10 16:38 jiayingz: If you do 64->32 its a different set of errors. 2008-04-10 16:38 willn, but it also fails with 64->32? 2008-04-10 16:39 jiayingz: http://code.google.com/p/zumastor/issues/detail?id=110#c2 2008-04-10 16:40 the short is, yes 2008-04-10 16:41 jiayingz: That is the general API, but it was only introduced in gcc-4.1 2008-04-10 16:43 So, we need to test for our gcc version, and I guess for older stuff we need to #include ? 2008-04-10 16:47 cbsmith, can we use signals instead to detect io_destroy? 2008-04-10 16:53 jiayingz: Hmm.. I am going to look at the kernel code to see what all happens when you close during io_getevents, but it didn't look like any signal was generated. 2008-04-10 16:54 jiayingz: testpaio doesn't exactly mask any signals, so I'm not too hopeful. 2008-04-10 16:56 cbsmith, can we send the aio_event_reader thread a signal in teardown? 2008-04-10 16:56 jiayingz: we could do a pthread_kill().... 2008-04-10 16:56 That's an interesting idea. 2008-04-10 17:22 I don't see anything in the aio codebase which wakes up during aio_evt_read from a signal or a context being destroyed. 2008-04-10 17:30 I guess if we spawned a process rather than a thread it might work more cleanly, but I'm not so sure we can get fork to share the right things for that to work. 2008-04-10 18:05 -!- flipz(~phlipz@adsl-63-202-13-187.dsl.snfc21.pacbell.net) has joined #zumastor 2008-04-10 18:29 ACTION grumbles somethings about the merits of using mementos instead of just having your native structures be packed in serialized form.... 2008-04-10 22:21 -!- phoenix24(kgusvgq@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-10 23:41 -!- phoenix24(ijh@h1119083.serverkompetenz.net) has joined #zumastor irc.oftc.net #zumastor log beginning Fri Apr 11 00:00:01 PDT 2008 2008-04-11 00:41 -!- phoenix24_(txu@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-11 01:41 -!- phoenix24(bqsp@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-11 02:42 -!- phoenix24(udfhw@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-11 04:19 -!- phoenix24(zzb@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-11 06:08 -!- tim_vimm(~Tim@216.237.95.243) has joined #zumastor 2008-04-11 06:09 -!- tim_vimm_(~Tim@216.237.95.243) has joined #zumastor 2008-04-11 08:28 good morning 2008-04-11 09:57 -!- jiayingz(~jiayingz@207.47.98.129.static.nextweb.net) has joined #zumastor 2008-04-11 10:22 hi jiayingz 2008-04-11 10:23 hi flipz 2008-04-11 10:24 hi everyone 2008-04-11 10:26 my attempt to skate over the 101 yesterday failed 2008-04-11 10:27 the sidewalk on the overpass was too narrow and steep 2008-04-11 10:27 Oh... I had visions of you trying to be Evil Kenival.... 2008-04-11 10:27 had to turn back, skating backwards to control the speed 2008-04-11 10:28 so I judge it impractical to skate from mountain view to the campus 2008-04-11 10:28 by that route at least 2008-04-11 10:28 skating on the campus itself is very nice 2008-04-11 10:32 flipz and cbsmith, any more thoughts on using aio in ddsnapd? 2008-04-11 10:32 yeah 2008-04-11 10:32 I think I'm going to simply things for the first crack by just using aio for applying changes from the journal. 2008-04-11 10:33 It's a nice, safe, first step. 2008-04-11 10:33 Oh, and I meant to ask... any reason we're not using signalfd instead of using a signal handler. 2008-04-11 10:33 yes, i think that is the simpler case. but we may need to look at journal replay to see if we handle that properly 2008-04-11 10:34 flipz, what do u think? 2008-04-11 10:34 jiayingz: We don't right now, but it seems trivial enough. The nice thing is that if you fail midway through, it's okay, you just replay the entire journal. 2008-04-11 10:35 So all we have to do is wait for all the io's to complete before clearing the journal. 2008-04-11 10:36 but ddsnap server may die before some journal writes complete. so some journal blocks may contain invalid data 2008-04-11 10:36 yeah, and that's when you replay the entire journal. 2008-04-11 10:36 oh wait... you mean before the journal is completely written. 2008-04-11 10:37 yes 2008-04-11 10:37 i think flipz has some pending changes on journal replay 2008-04-11 10:37 I'm proposing for our first pass we don't do aio for the actual writing of the journal. I think aio won't provide as much of a performance win there anyway. 2008-04-11 10:38 We'll do that with aio later, once we've got this initial bit in. 2008-04-11 10:38 not much performance win since it won't change synchronous io response time. but it will increase ddsnapd parallelism 2008-04-11 10:39 currently, under heavy loads, 'ddsnap status' will take a long time because it needs to wait when ddsnap server is busy at processing write requests in its queue 2008-04-11 10:41 aio will help to solve this problem if we can get ddsnapd out of the waiting time for those writes 2008-04-11 10:41 Yeah, I'm just figuring writing to the journal is pretty quick because you probably only have one seek to do it. 2008-04-11 10:41 I don't know if we've timed it, but I bet we write to the journal *way* faster than we commit the journal. 2008-04-11 10:42 by 'write to the journal', you mean writing to the on-disk journal? 2008-04-11 10:42 yes 2008-04-11 10:43 why do you think it will be much faster? 2008-04-11 10:43 right now ddsnapd is tied up while it both writes to the journal and commits the journal. With the aio patch I'm putting together, ddsnapd would only be tied up until the journal write is done. 2008-04-11 10:44 Because when you write to the journal it's one contiguous block, so only one seek. 2008-04-11 10:44 did I get that wrong? 2008-04-11 10:45 i think commit block is right after those journal blocks 2008-04-11 10:45 ACTION looks at the code again. 2008-04-11 10:46 cbsmith, currently journal commit only one transaction at a time and only the last transaction is replayed 2008-04-11 10:47 in order for aio to be useful there, multiple parallel journal transactions need to be implemented, which implies analyzing dependencies between them 2008-04-11 10:47 flipz: Ah, I see, and by definition that means there is only one commit block. 2008-04-11 10:47 not so easy 2008-04-11 10:47 there is only one commit block that ever needs to be replayed 2008-04-11 10:47 now I get it. 2008-04-11 10:48 until my bitmap optimization goes in 2008-04-11 10:48 So the real win would be to allow multiple transactions at once. 2008-04-11 10:48 then, only the allocations are replayed from the full journal 2008-04-11 10:48 it would, but dependencies need to be analyzed, and adding the same buffer to multiple transactions needs to be handled 2008-04-11 10:48 yeah, I'm thinking through how you'd handle that algorithmically right now. 2008-04-11 10:49 asynchronous copyout would be an easier place to start 2008-04-11 10:49 developing a fully ansynchronous journal is a challenge 2008-04-11 10:49 Okay I'll work on the copyout then. 2008-04-11 10:50 flipz, can we just replay those transactions in order instead of figuring out their dependencies? 2008-04-11 10:50 Looks like a combination of "copyout" and "finish_copyout" need to be touched for that, right? 2008-04-11 10:50 jiayingz, yes 2008-04-11 10:50 jiayingz: Yes, that works for replay, but what if everything goes well? If you have two writes to the same place in flight at the same time... 2008-04-11 10:50 cbsmith, :) 2008-04-11 10:51 jiayingz, however you need to figure out to do when a buffer is added to a new transaction, and is also in a transaction that has not been comitted yet 2008-04-11 10:51 we need to implement locking/wait to prevent that 2008-04-11 10:51 jiayingz: I'm not sure that is actually what you want to do. It could be that the smart thing to do is to cancel the older io, and then mark *both* transactions as complete when the later one is done. 2008-04-11 10:52 for using aio in copyout 2008-04-11 10:52 jiayingz, yes, actually dependency processing needs to be implemented 2008-04-11 10:53 cbsmith, your approach sound more efficient, but i don't know if there is any race there 2008-04-11 10:53 cbsmith, signalfd did not exist at the time I implemented ddsnapd 2008-04-11 10:54 flipz: Should we use it now? 2008-04-11 10:54 the current approach works fine 2008-04-11 10:54 jiayingz: Yeah, that's why I'm not sure. ;-) 2008-04-11 10:55 jiayingz, implementing aio for buffer reads sounds like a very good idea 2008-04-11 10:55 much easier than journal writes 2008-04-11 10:56 flipz, i guess so. we can reuse the existing locking/wait code 2008-04-11 10:56 flips: buffer *reads*? 2008-04-11 10:56 maybe :-) 2008-04-11 10:56 something similar 2008-04-11 10:56 yes, reads 2008-04-11 10:56 currently they have to wait on journal writing 2008-04-11 10:56 i think flips means reading the old origin data during copyout 2008-04-11 10:56 not efficient 2008-04-11 10:56 ACTION blinks 2008-04-11 10:57 I mean reading metadata for "status" 2008-04-11 10:57 but metadata should be in memory 2008-04-11 10:57 flips: Oh that's an interesting idea. I like it, because we can't corrupt the journal. 2008-04-11 10:57 jiayingz, only part of the metadata normally 2008-04-11 10:57 jiayginz, and reading with aio allows better disk elevator performance 2008-04-11 10:58 we can have aio readahead 2008-04-11 10:58 flipz, i though normal the whole btree is in memory 2008-04-11 10:59 if that is not the case, we should optimize that 2008-04-11 11:01 jiayingz, there is only about 128 MB of buffer cache, often smaller than the snapshot store metadata 2008-04-11 11:01 but the big win is being able to do readahead 2008-04-11 11:01 to reduce seeking 2008-04-11 11:01 flipz, yes. but we can set buffer cache to larger size 2008-04-11 11:02 and we are still not quite sure how btree grows 2008-04-11 11:02 that doesn't help with readahead 2008-04-11 11:02 which we need to speed up status 2008-04-11 11:03 why isn't status in its own function? 2008-04-11 11:04 i agree that speeding up 'ddsnap status' is important, but i think using aio to speed up copyout and journal writes can provide bigger win 2008-04-11 11:05 jiayingz: I think you are both right, but here's the thing: status is the safer one. We do that first, and then do the copyout. 2008-04-11 11:05 jiayingz: The cool thing about status is you can't corrupt anything. 2008-04-11 11:05 even if you screw up, the worst thing that happens is that you get status. 2008-04-11 11:05 cbsmith, i agree 2008-04-11 11:06 jaiyingz, the wisest place to start using aio is in copyout 2008-04-11 11:06 because it is the easiest 2008-04-11 11:07 status is a close second 2008-04-11 11:07 it requires implementing per-buffer read locks 2008-04-11 11:07 flipz, i thought you think status is the easiest :) 2008-04-11 11:07 copyout is the easiest 2008-04-11 11:07 because it doesn't need any locking 2008-04-11 11:07 it does. 2008-04-11 11:07 it only needs a wait queue for write transactions 2008-04-11 11:07 that is not a lock 2008-04-11 11:08 you need a lock to tell later write requests to wait 2008-04-11 11:09 flipz, by 'copyout', u mean read in copyout? 2008-04-11 11:10 read and write 2008-04-11 11:10 jiayingz, that is not a lock, it is a queue 2008-04-11 11:11 both are synchronizers, indeed 2008-04-11 11:11 blink 2008-04-11 11:12 for writing to snapshot, we need a wait queue to synchronize snapshot write and read 2008-04-11 11:12 that is true 2008-04-11 11:13 i think using 'ddsnap status' is the simpliest. it does not need any synchronization 2008-04-11 11:13 then it is copyout 2008-04-11 11:13 then journal writes 2008-04-11 11:13 jiayingz, it does need synchronization 2008-04-11 11:14 because a buffer may be requested to handle some other request during the readahead 2008-04-11 11:14 or written to while it is being read asynchronously 2008-04-11 11:14 but ddsnapd is not processing other requests 2008-04-11 11:14 jyaingz, then you are correct 2008-04-11 11:14 but it should be processing other requests 2008-04-11 11:15 ok. so u r thinking to make ddsnap server processes 'ddsnap status' asynchronously 2008-04-11 11:15 yeah, I think the whole idea would be to free it up to complete other requests, right? 2008-04-11 11:15 that would be nice, but tricky 2008-04-11 11:16 jiayingz, even if status is not asynchronous, aio for readahead will still be a win 2008-04-11 11:16 but... copyout aio will be a better win overall 2008-04-11 11:16 so flip a coin? 2008-04-11 11:16 i thought that is ur intention 2008-04-11 11:16 either is ok with me 2008-04-11 11:16 :) 2008-04-11 11:16 got to start somewhere 2008-04-11 11:17 start with journal aio, for a real challenge ;) 2008-04-11 11:17 just using aio for readahead is the easiest 2008-04-11 11:17 i always prefer to starting with something simpler :) 2008-04-11 11:17 good choice 2008-04-11 11:17 quit squabbling. I'm working on the status one right now. 2008-04-11 11:18 I'm going to move it in to a separate function too, if that's all right with you two. 2008-04-11 11:18 cbsmith, why not u just try using aio for readahead in 'ddsnap status' 2008-04-11 11:18 jiayingz: yeah, that's what I'm doing. 2008-04-11 11:18 fine with me 2008-04-11 11:18 we can see how much performance gain it provides 2008-04-11 11:18 then we can try copyout 2008-04-11 11:19 yup 2008-04-11 11:19 I expect it'll be a huge win with RAID. 2008-04-11 11:19 Probably pretty good on my laptop too. 2008-04-11 11:21 I guess I should do a patch just with moving it out to a function. 2008-04-11 11:23 zcommits: zumastor b0.8.0 r1533 build failure 1 2008-04-11 11:43 -!- dank(~chatzilla@cpe-76-90-56-73.socal.res.rr.com) has joined #zumastor 2008-04-11 11:49 jiayingz: Did that last patch I sent out actually look like a proper MIME attachment? 2008-04-11 11:51 I just approved it - google groups thought it was spam. It did look funny. I haven't seen it show up in my inbox or the archives yet. 2008-04-11 11:57 cbsmith, yes. actually the last two are both 2008-04-11 11:57 dank: What looked "funny" about it? 2008-04-11 12:03 dank: I am not seeing your reply... even in the spam folder 2008-04-11 13:29 -!- flipz(~phlipz@216.239.45.19) has joined #zumastor 2008-04-11 13:43 shapor, there? 2008-04-11 13:58 zcommits: [zumastor commit] r1534 - in trunk/cbtb: host-scripts host-setup 2008-04-11 14:13 Interesting, the AIO list got me a response on the destroy/io_getevents race. 2008-04-11 14:17 cbsmith, can u use pthread_kill upon exit to avoid the race 2008-04-11 14:17 jiayingz: nope :-) 2008-04-11 14:17 jiayingz: But there is a proposed patch. 2008-04-11 14:17 jiayingz: You might be able to submit the equivalent of a noop IO to get it to wake up. 2008-04-11 14:17 cbsmith, why do u need io_destroy? 2008-04-11 14:17 jiayingz: Need to clean up the resources. 2008-04-11 14:18 i c 2008-04-11 14:19 the io to destroy should already be returned from io_getevents 2008-04-11 14:20 io_getevents is happening in a different thread. 2008-04-11 14:20 and we need some way to tell that thread to shut down. 2008-04-11 14:20 I guess we could detach the thread, but that seems a little creepy. 2008-04-11 14:21 yes. why pthread_kill does not work? 2008-04-11 14:24 jiayingz: because the io_getevents call does not appear to return because of a signal. 2008-04-11 14:25 after io_getevents returns after timeout, can it catch the signal? 2008-04-11 14:25 oh sure 2008-04-11 14:25 Which is why I left in the timeout 2008-04-11 14:25 But we don't want to have a timeout. 2008-04-11 14:26 timeout is just a figleaf over polling. ;-) 2008-04-11 14:37 why do we stick svnversion into a variable instead of just calling svnversion when we need it? 2008-04-11 14:52 flips: during Make? 2008-04-11 14:52 flips: I thought we only called it once? 2008-04-11 14:57 cbsmith, storing it in a variable does not make sense 2008-04-11 14:57 false economy 2008-04-11 14:57 let's just get rid of the variable and call the utility everywhere we need it 2008-04-11 14:57 get rid of some bogus code and variables 2008-04-11 14:57 flips: Where is the variable? 2008-04-11 14:57 all over the place 2008-04-11 14:57 sucks 2008-04-11 14:58 flips: At least in the ddsnap Makefile I don't see a variable... 2008-04-11 14:58 I do 2008-04-11 14:58 SVNREV 2008-04-11 14:58 a file actually 2008-04-11 14:58 worse 2008-04-11 14:58 @echo '#define SVN_VERSION "'`cat SVNREV || svnversion .`'"' >> $@ 2008-04-11 14:59 oh, IIRC that was to deal with svnversion bugs. 2008-04-11 14:59 such as? 2008-04-11 14:59 I honestly don't remember. Wasn't me who did it. 2008-04-11 14:59 it would be good to know 2008-04-11 14:59 Yeah, IIRC it was goobuntu specific. 2008-04-11 14:59 fsck 2008-04-11 14:59 fix goobuntu 2008-04-11 15:00 -!- jiayingz_(~jiayingz@207.47.98.129.static.nextweb.net) has joined #zumastor 2008-04-11 15:00 instead of spamming the repo 2008-04-11 15:00 I could be wrong about that. 2008-04-11 15:01 if it is necessary to alias svnversion when building on a broken system, do that 2008-04-11 15:02 actually, ddsnap should cat a generic file REVISION with is set outside, and there should be no other mention of revision in the ddsnap makefile 2008-04-11 15:02 so REVISION could be set by hand or whatever 2008-04-11 15:03 right now, if you build ddsnap without building all of zumastor you get an error 2008-04-11 15:03 that is because the REVISION file is not versions 2008-04-11 15:03 versioned 2008-04-11 15:03 it should be 2008-04-11 15:03 ok, let's try it that way, small patch coming 2008-04-11 15:03 we don't need to go hunting all the SVNREV gack just now 2008-04-11 15:04 cbsmith, your attmpt to fix it is the right spirit, just needs readjustment 2008-04-11 15:05 flipz: again, not my fix. ;-) 2008-04-11 15:06 sorry, missed that 2008-04-11 15:06 seems obvious there should be no mention of svn in the ddsnap makefile 2008-04-11 15:09 What, isn't Make completely integrated with version control? Don't you always build with a version of the source checked out from SVN? ;-) 2008-04-11 15:09 Actually, I guess that'd be a reason to have the file. That way you could provide tarballs of the source with the file in it and thereby avoid the problems with svnversion not being there. 2008-04-11 15:13 yes 2008-04-11 15:13 in fact, in the ddsnap case, it's the call to svnversion that needs to go 2008-04-11 15:14 and SVNREV -> REVISION, checking into the svn tree 2008-04-11 15:14 checked in 2008-04-11 15:14 ACTION would love that 2008-04-11 15:14 now, it's immediately going to get out of sync 2008-04-11 15:14 so it will really mean "the last time fully built" 2008-04-11 15:14 which is not such a bad thing 2008-04-11 15:17 You know, I was trying to do something like that with the zumastor script and got nailed for letting it get out of sync. 2008-04-11 15:20 it's a philosphical question 2008-04-11 15:21 current solution is clearly wrong 2008-04-11 15:23 from the git point of view, a REVISION file is clearly wrong 2008-04-11 15:23 because the revision is determined fully by the hash of the file content 2008-04-11 15:23 or directory in this case 2008-04-11 15:24 hmm 2008-04-11 15:24 hey, and to think I wanted to use a hash of the zumastor script as the id.... ;-) 2008-04-11 15:24 I'm just thinking about the truth of my claim 2008-04-11 15:24 actually, the revision number is separate from the object hashes 2008-04-11 15:25 can captures the sense of the full tree rev 2008-04-11 15:25 there is not one of these things at each node of the tree 2008-04-11 15:25 it would be bad if there were (too much updating) 2008-04-11 15:26 but it would be trivial to introduce the notion of a subtree object id, computed as the hash of all objects in the tree 2008-04-11 15:26 or xor even, which would have awfully nice properties, but I digress 2008-04-11 15:27 got to get that SVNREV and svnversion line out of ddsnap, convert to something else 2008-04-11 15:27 environment variable maybe 2008-04-11 15:30 $ ./ddsnap --version 2008-04-11 15:30 ddsnap revision "exported" 2008-04-11 15:30 built on Fri Apr 11 18:29:39 EDT 2008 by daniel@marine 2008-04-11 15:30 on two lines... that is bad 2008-04-11 15:31 calling svnversion is wrong anyway, this might be built from a version not checked in 2008-04-11 20:37 -!- tim_vimm(~Tim@216.237.95.243) has joined #zumastor 2008-04-11 22:35 zcommits: zumastor b0.8.0 r1534 build failure 255 irc.oftc.net #zumastor log beginning Sat Apr 12 00:00:01 PDT 2008 2008-04-12 05:58 -!- tim_vimm(~Tim@216.237.95.243) has joined #zumastor 2008-04-12 11:58 -!- MaZe(~MaZe@65.42.208.133) has joined #zumastor 2008-04-12 14:34 -!- jiayingz_(~jiayingz@cpe-76-167-221-105.socal.res.rr.com) has joined #zumastor 2008-04-12 21:44 -!- cbsmith(~user@adsl-99-165-23-73.dsl.lsan03.sbcglobal.net) has joined #zumastor 2008-04-12 21:45 -!- jiayingz_(~jiayingz@cpe-76-167-221-105.socal.res.rr.com) has joined #zumastor 2008-04-12 23:09 -!- MaZe(~MaZe@c-24-6-86-168.hsd1.ca.comcast.net) has joined #zumastor 2008-04-12 23:13 flips: I'm back in MTV 2008-04-12 23:15 -!- jiayingz_(~jiayingz@cpe-76-167-221-105.socal.res.rr.com) has joined #zumastor irc.oftc.net #zumastor log beginning Sun Apr 13 00:00:01 PDT 2008 2008-04-13 01:46 -!- phoenix24(stsbym@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-13 02:48 -!- phoenix24(dxfmfao@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-13 02:48 ew! isp sucks! 2008-04-13 03:51 -!- phoenix24(jof@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-13 04:51 -!- phoenix24_(qzqmft@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-13 06:31 -!- phoenix24(hujjha@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-13 07:39 -!- phoenix24(znw@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-13 09:04 -!- phoenix24(mwfwbkv@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-13 10:04 -!- phoenix24_(efjojjl@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-13 10:33 -!- MaZe(~MaZe@c-24-6-86-168.hsd1.ca.comcast.net) has joined #zumastor 2008-04-13 11:05 -!- jiayingz_(~jiayingz@cpe-76-167-221-105.socal.res.rr.com) has joined #zumastor 2008-04-13 11:21 -!- phoenix24(zpngig@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-13 11:44 -!- MaZe(~MaZe@c-24-6-86-168.hsd1.ca.comcast.net) has joined #zumastor 2008-04-13 21:48 -!- phoenix24(faguixo@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-13 23:17 now to try my first actual merge in git 2008-04-13 23:18 merge the git repo from my laptop with the hacked repo on my desktop 2008-04-13 23:18 this is supposed to work ;-/ 2008-04-13 23:26 its not? irc.oftc.net #zumastor log beginning Mon Apr 14 00:00:01 PDT 2008 2008-04-14 01:40 -!- erwan_taf(~erwan@81.80.43.67) has joined #zumastor 2008-04-14 01:43 -!- phoenix24(kmjzfs@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-14 03:00 -!- phoenix24(evilb@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-14 03:41 -!- MaZe(~MaZe@c-24-6-86-168.hsd1.ca.comcast.net) has joined #zumastor 2008-04-14 04:05 -!- phoenix24(bxw@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-14 04:23 -!- erwan_taf(~erwan@81.80.43.67) has joined #zumastor 2008-04-14 04:24 -!- willn(~wan@pinball.ccs.neu.edu) has joined #zumastor 2008-04-14 04:24 -!- cbsmith(~user@adsl-99-165-23-73.dsl.lsan03.sbcglobal.net) has joined #zumastor 2008-04-14 04:24 -!- jiayingz(~jiayingz@207.47.98.129.static.nextweb.net) has joined #zumastor 2008-04-14 04:24 -!- natalie(~nataliep@207.47.98.129.static.nextweb.net) has joined #zumastor 2008-04-14 04:24 -!- zbot(wan@209.67.252.126) has joined #zumastor 2008-04-14 04:24 -!- DavidSev(david@lolcat.mercenariesguild.net) has joined #zumastor 2008-04-14 04:24 -!- flips(~phillips@phunq.net) has joined #zumastor 2008-04-14 04:24 -!- MaZe(~MaZe@c-24-6-86-168.hsd1.ca.comcast.net) has joined #zumastor 2008-04-14 04:24 -!- shapor(~shapor@yzf.shapor.com) has joined #zumastor 2008-04-14 04:24 -!- phoenix24(bxw@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-14 04:24 -!- pgquiles_(~pgquiles@25.Red-88-16-37.dynamicIP.rima-tde.net) has joined #zumastor 2008-04-14 04:39 -!- pgquiles__(~pgquiles@62.43.226.52.static.user.ono.com) has joined #zumastor 2008-04-14 04:48 -!- pgquiles_(~pgquiles@25.Red-88-16-37.dynamicIP.rima-tde.net) has joined #zumastor 2008-04-14 05:34 -!- CompBrain(~wan@pinball.ccs.neu.edu) has joined #zumastor 2008-04-14 06:25 -!- phoenix24(fetwb@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-14 06:51 Is the SVN repository inaccessible to anybody else too ? 2008-04-14 08:29 phoenix24 it just worked for me 2008-04-14 09:19 -!- phoenix24(uznne@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-14 10:08 ACTION blinks 2008-04-14 10:48 good morning 2008-04-14 10:49 shapor, I expect git merge works, yes 2008-04-14 10:49 but I ended up just taking a patch and applying it, because I would have had to install apache on the laptop and mess with httpd.conf otherwise 2008-04-14 10:49 or set up nfs 2008-04-14 10:57 flips won't changing fd_size to size_t cause a bug when its 32 bit and you ioctl BLKGETSIZE64 ? 2008-04-14 10:57 why should it? 2008-04-14 10:57 the result needs to be 64 bits yes 2008-04-14 10:57 let me check 2008-04-14 10:58 you're right, but see "must compile with size_t = 64 bits" 2008-04-14 10:58 if we must compile with size_t = 64, why not just use u64 then? 2008-04-14 10:59 because it's frigging ugly the way it was done? 2008-04-14 10:59 its explicitly 64 because the code is that way 2008-04-14 10:59 seems like a silly thing to change 2008-04-14 11:00 I think I changed it in the process of moving fd_size into diskio.c 2008-04-14 11:00 everything else cascaded from that 2008-04-14 11:01 the only thing that needs to change to make it correct is the ioctl number needs to be dependent on size_t size 2008-04-14 11:01 or it can use a temporary variable 2008-04-14 11:01 in order to support 32 bit file ops compilation 2008-04-14 11:02 which is theoretical anyway 2008-04-14 11:02 %zi and size_t is less code, and is self-documenting 2008-04-14 11:02 so it is better 2008-04-14 11:02 why should we not do it? 2008-04-14 11:03 i'm not saying we shouldn't 2008-04-14 11:03 I'm not saying we should ;) 2008-04-14 11:03 its not the thing which makes my eyes bleed the most, thats all ;) 2008-04-14 11:04 the "U64FMT" doesn't make your eyes bleed? 2008-04-14 11:04 it makes mine bleed, profusely 2008-04-14 11:04 theres much worse in ddsnap.c ;) 2008-04-14 11:05 is there? 2008-04-14 11:05 think about the evil person who coded this 2008-04-14 11:05 U64FMT is evil, but then again, printf makes my eyes bleed. ;-) 2008-04-14 11:05 indended #define U64FMT "%li" 2008-04-14 11:05 intended 2008-04-14 11:05 pure evil 2008-04-14 11:06 flips: I believe, btw, that this is a common idiom amongst folks at G. 2008-04-14 11:06 yikes 2008-04-14 11:06 amonst which people? 2008-04-14 11:06 not sane ones 2008-04-14 11:07 yeah, makes you wanna cry. 2008-04-14 11:07 The sane ones use C++ style formatting, I'm sure. ;-) 2008-04-14 11:08 shapor, I will fix fd_size to be correct wrt size_t 2008-04-14 11:08 good catch 2008-04-14 11:08 yeah i think that would have ended up scribbling 2008-04-14 11:08 I don't get it though, why must we compile in a world where size_t=64 bits? 2008-04-14 11:10 cbsmith, because we live in a world where we do posix io on volumes that are bigger than 32 bits 2008-04-14 11:11 flips: But POSIX IO works fine in that circumstance, even if size_t = 32 bits. That's what you have off_t. 2008-04-14 11:12 cbsmith, oops I think you are right 2008-04-14 11:14 flips: I'm pretty sure I am. POSIX's approach is you can't read more than 2^size_t in to memory... which makes a lot of sense, but you can use off_t to work with files as large as 2^off_t. Of course, if you don't have large file support, off_t might be 32-bits as well, but you also are probably about to get smacked around by a few people. 2008-04-14 11:14 yah, throw this patch away 2008-04-14 11:15 I don't think we will be mmaping more than 4 gig 2008-04-14 11:15 flips: Yeah, best not to plan on it. ;-) 2008-04-14 11:16 and of course, no formatting operator available for off_t 2008-04-14 11:16 this used to be all written by casting file offset values to long long, and some artist decided to change that 2008-04-14 11:16 now i see why i was confused about the whole patch 2008-04-14 11:16 it was pointless :P 2008-04-14 11:16 I think I will put it back that way 2008-04-14 11:16 shapor: yup 2008-04-14 11:17 ah, not pointless 2008-04-14 11:17 U64FMT is evil 2008-04-14 11:17 flips: Yeah, I'm not sure who got rid of all the long long casts that I put in there. 2008-04-14 11:17 I just put the wooden stake in the wrong place 2008-04-14 11:17 flips: Really, not having typesafe formatting functions is evil. ;-) 2008-04-14 11:18 anyway, fd_size should not be size_t, it should be off_t 2008-04-14 11:18 or 764 2008-04-14 11:18 ur u64 2008-04-14 11:18 and then the name should be fdsize64 2008-04-14 11:18 so I'll do that instead 2008-04-14 11:18 kay? 2008-04-14 11:19 not kill U64FMT just now 2008-04-14 11:20 kk 2008-04-14 11:20 I've been periodically going through the code and removing any U64FMT's I see. 2008-04-14 11:20 good practice 2008-04-14 11:22 flips: We might want to stick with u64 in a number of cases as some of those bits eventually get shuffled over the wire. 2008-04-14 11:22 naturally 2008-04-14 11:29 off_t fdsize64(int fd) 2008-04-14 11:29 { 2008-04-14 11:29 uint64_t *bytes; 2008-04-14 11:29 struct stat stat; 2008-04-14 11:29 if (fstat(fd, &stat) == -1) 2008-04-14 11:29 return -1; 2008-04-14 11:29 if (S_ISREG(stat.st_mode)) 2008-04-14 11:29 return stat.st_size; 2008-04-14 11:29 if (ioctl(fd, BLKGETSIZE64, bytes)) 2008-04-14 11:30 return -1; 2008-04-14 11:30 return bytes; 2008-04-14 11:30 } 2008-04-14 11:30 ok? 2008-04-14 11:30 whoops 2008-04-14 11:30 should return uinit64_t 2008-04-14 11:30 uint64_t 2008-04-14 11:33 flips: Depends on what you want to expose: what the OS is gonna give you or what you are going to send over the wire. I'd argue uint64_t is the right one though. 2008-04-14 11:33 it's the least intrusive change for now 2008-04-14 11:34 later, unint64_t fdsize64 -> off_t fdsize 2008-04-14 11:34 kk 2008-04-14 11:34 with associated U64FMT removal 2008-04-14 11:40 -!- sumit(~sumit@snsl.engr.uconn.edu) has joined #zumastor 2008-04-14 11:40 -!- sumit(~sumit@snsl.engr.uconn.edu) has left #zumastor 2008-04-14 11:42 why do we compile buffer.c separately when only ddsnapd.c uses it? 2008-04-14 11:50 flips: because it's a good thing to do. :-) 2008-04-14 11:50 why is it a good thing to do? 2008-04-14 11:50 flips: Plus, if you ever move in to kernel space, most of buffer.c is redundant. 2008-04-14 11:51 flips: nice, clean interface. No need for tight coupling with ddsnapd 2008-04-14 11:51 it can be compiled as a separate source without generating a .o 2008-04-14 11:51 and without changing the interface 2008-04-14 11:51 flips: Oh I see. 2008-04-14 11:51 probably I didn't know how to do that when I wrote the prototype 2008-04-14 11:51 it was me who did that 2008-04-14 11:51 flips: The main reason to make the .o is so you don't compile buffer.c every time ddsnapd gets tweaked (which in my case, is pretty often). 2008-04-14 11:52 point 2008-04-14 12:00 flips: So this AIO stuff.. the one problem I'm seeing is that for best efficiency, you really want to do a breadth first search, only that is not so nice on the memory consumption. I'm currently coding it up as depth first, still, but it limits the number of IO's you can have in flight at once fairly substancially. 2008-04-14 12:00 you will need to limit the number of ios in flight 2008-04-14 12:01 flips: Sure. This is just going to limit it pretty severely. 2008-04-14 12:01 to (much) less than the buffer size available 2008-04-14 12:01 flips: Right now I'm basically limited to the number of leaves pointed to by a single node. 2008-04-14 12:01 remember, I thought it would be easiest to start with copyout 2008-04-14 12:01 flips: So things like thousands of in-flight IO's don't seem likely. 2008-04-14 12:02 flips: Yeah, but it's safest to do status. ;-) 2008-04-14 12:02 it has to be more than unlikely, it has to be proved impossible 2008-04-14 12:02 status that consumes too much memory is not safe and will deadlock 2008-04-14 12:02 "unlikely" was more a joke. ;-) 2008-04-14 12:03 Just wanted to make sure our expectations were the same. 2008-04-14 12:27 -!- MaZe(~MaZe@216-239-45-4.google.com) has joined #zumastor 2008-04-14 12:47 ddt-zumastor: Handle the loop limit of change_bits correctly. Fix missing brelse for flag=0. 2008-04-14 13:17 Hooray for whitespace fun 2008-04-14 13:32 -!- phoenix24(neof@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-14 14:33 I've got a draft patch for disabling compression 2008-04-14 14:55 zissues: Issue 74 in zumastor: Provide option to skip compression during replication 2008-04-14 14:59 is it possible to use zumastor as a rsync substitute, ie. no past snapshots, just the current one? how much snapshots should I use for that --daily 0? --daily 1? 2008-04-14 15:00 1 I think... 2008-04-14 15:00 You mean just to replicate a volume? 2008-04-14 15:01 willn: yes, just replication 2008-04-14 15:16 In theory you don't need to turn on a replication schedule 2008-04-14 15:16 as snapshots are created for replication purposes 2008-04-14 15:17 pgquiles_, we are thinking to add in zumastor-howto on how to use zumastor as rsync replacement 2008-04-14 15:19 you can send us a howto patch if you already have something in your mind 2008-04-14 15:20 jiayingz: I have not tried it yet but I will tomorrow. I'll try to write something down. 2008-04-14 15:20 pgquiles_, that is great! :) 2008-04-14 15:24 that's a nice use case 2008-04-14 15:25 or usage mode perhaps I should say 2008-04-14 15:25 bio backup is no fun for anyone 2008-04-14 15:25 willn, zbot is yours? 2008-04-14 15:25 >--< <-- thats my data pipe 2008-04-14 15:25 flips: Yea 2008-04-14 15:26 nice one 2008-04-14 15:26 bio backup? 2008-04-14 15:26 Things were going so well. 2008-04-14 15:26 what is bio backup? 2008-04-14 15:27 slow io 2008-04-14 15:27 oh 2008-04-14 15:27 patch needed from me 2008-04-14 15:27 I was doing the randomfiles generation from /dev/urandom 2008-04-14 15:27 right after current optimization patch 2008-04-14 15:27 onto 50G lvm volume ontop of 2xraid6 2008-04-14 15:27 with zumastor in the mix 2008-04-14 15:28 is it the socket sk_alloc problem? 2008-04-14 15:29 damn machine has too many processes, sysrq fills the ringbuffer 2008-04-14 15:30 *overflows 2008-04-14 15:32 ddsnap inflight bio vecs 169963, pending requests: 349, query requests 169614, release requests 0, locked requests 0 2008-04-14 15:32 dd is hung waiting 2008-04-14 15:34 I seem to recall it was because of a socket write needing to get memory 2008-04-14 15:34 but some evidence would help 2008-04-14 15:34 willn, is it compiled with maximum printk buffer size? 2008-04-14 15:36 the kernel? 2008-04-14 15:37 yes 2008-04-14 15:37 does not appear so 2008-04-14 15:38 do we own the config? if so, setting printk size up to max is good, and make sure frame pointers are enabled 2008-04-14 15:38 It's a ubuntu derrived configuration 2008-04-14 15:39 I stopped a process and finally the inflight bio vecs are decreasing 2008-04-14 15:40 well we compile the kernel of course, so let's make those config changes 2008-04-14 15:40 nice clue 2008-04-14 15:40 This is the kernel we're having the PPA build 2008-04-14 15:40 ppa? 2008-04-14 15:40 launchpad build service 2008-04-14 15:41 willn: how different from ubuntu's is the config in the PPA kernels? 2008-04-14 15:41 "personal package archive" 2008-04-14 15:41 sounds like a use of time to me 2008-04-14 15:42 +CONFIG_DM_DDSNAP=m 2008-04-14 15:42 ;) 2008-04-14 15:42 willn: :-) 2008-04-14 15:42 flips: pardon? 2008-04-14 15:42 translation, I still haven't figured out what it's for 2008-04-14 15:42 You send them debian source packages 2008-04-14 15:42 you wait, they send you binary packages for 3 arch's 2008-04-14 15:43 ok, that sounds useful 2008-04-14 15:43 we do need to be able to adjust the kernel config though 2008-04-14 15:43 It is going to be some time before this gets flushed. 2008-04-14 15:43 ACTION goes to work on something else 2008-04-14 15:43 willn: are there prebuilt ubuntu packages for zumastor-stable anywhere? 2008-04-14 15:44 Nothing at the PPA is a release branch at the moment 2008-04-14 15:45 Sometime if i've got time I plan to change snapshot builds to TESTING and make sure we have release branch RELEASE packages 2008-04-14 15:45 what about having zumastor and zumastor-unstable packages? 2008-04-14 15:45 We could, though the only way to do that is to start another launchpad team 2008-04-14 15:46 another launchpad team? what for? 2008-04-14 15:46 they don't do multiple ppas for one person/team 2008-04-14 15:46 oh, you mean seperate package names? 2008-04-14 15:46 yes 2008-04-14 15:46 it's the easiest way 2008-04-14 15:46 ah. That seems unclea 2008-04-14 15:46 unclean* 2008-04-14 15:47 I think it's exactly the opposite: when you install the zumastor-unstable package, you know you will be on the bleeding edge 2008-04-14 15:47 the package name shouts "don't install me!" at you :-) 2008-04-14 15:48 Yeah, that's reasonable. 2008-04-14 15:48 Have to make sure we get the appropriate conflicts in there 2008-04-14 15:48 yes 2008-04-14 15:49 a "Provides: zumastor" line in zumastor-unstable should also be added 2008-04-14 15:50 flips: have you seen this http://research.swtch.com/2008/04/backups-heal-thyself.html ? 2008-04-14 15:50 I have seen venti before 2008-04-14 15:50 it looks interesting 2008-04-14 15:51 seems like a good idea, one we've had independently 2008-04-14 15:51 identical blocks could/should be merged in the snapshot store 2008-04-14 15:51 even similar blocks 2008-04-14 15:52 it's always scary to rely on never getting a collision from sha1 2008-04-14 15:53 indeed 2008-04-14 15:53 though I know the argument - the change of a collision is much less than the chance of a hardware failure 2008-04-14 15:53 chance I mean 2008-04-14 15:53 chance is not the issue 2008-04-14 15:53 the real problem are users 2008-04-14 15:53 ? 2008-04-14 15:54 what happens if some user knows how to create a sha1 collision and writes to the venti storage? 2008-04-14 15:54 nobody knows that 2008-04-14 15:54 only 130k reqs to go... 2008-04-14 15:54 nobody knows that *yet* 2008-04-14 15:54 and even if they did, it is trivial to prevent 2008-04-14 15:55 anyway, I will read the whitepaper 2008-04-14 15:56 I just read the ext3_cow whitepaper and wonder why this has not gone upstream 2008-04-14 15:56 when I get time I will try to facilitate that 2008-04-14 15:56 I have not read anything about venti but the link I provided 2008-04-14 15:56 nilfs looks interesting, too 2008-04-14 15:57 they released 2.0 a couple of weeks ago 2008-04-14 15:57 "You are roughly 2^90 times more likely to win a U.S. state lottery and be struck by lightning simultaneously than you are to encounter [an accidental sha1 collision]" 2008-04-14 15:57 these filesystems really ought to go upstream 2008-04-14 15:58 I suppose people think the same aboud ddsnap 2008-04-14 15:58 though I want writes to be better optimized and to be using ddsetup before offering for upstream 2008-04-14 15:59 author of the above 2^90 claim needs more education in statistics 2008-04-14 16:00 but I have been convinced of the low probability argument by people with better math grounding in the past 2008-04-14 16:00 all the same, I've always wanted to try my hand at changing these sha1 algorithms to use a smaller, faster hash with classic collision detect/resolution 2008-04-14 16:01 the problem is, to detect a collision you have to compare against the original content 2008-04-14 16:02 in a content hash that is 2008-04-14 16:02 so what might work is, small hashes that index against full sha1 hashes 2008-04-14 16:02 with classic collision resolution 2008-04-14 16:03 I am willing to trust sha1, I just doing like using such big keys for the primary index 2008-04-14 16:04 as for attempts to compromise the sha1 in venti, the sha1 hash can include an inode-dependent secret that the attacker cannot possibly know, so does not know how to attack the hash internally 2008-04-14 16:04 this works for a system like venti, which hides the filesystem metadata from the user 2008-04-14 16:05 how to apply it to git, which does not hide tis metadata is a tough question 2008-04-14 16:06 put that one on the back burner 2008-04-14 16:06 not the most urgent question, by several orders of magnitude 2008-04-14 16:49 -!- MaZe(~MaZe@c-24-6-86-168.hsd1.ca.comcast.net) has joined #zumastor 2008-04-14 17:36 jiayingz: If you have a minute could you peek my no-compression patch? 2008-04-14 17:59 willn, i am looking at it 2008-04-14 18:05 jiayingz: Thanks! 2008-04-14 18:11 Ugh. Hanging again. 2008-04-14 18:11 A combination of unpacking kernels and RandomFiles.py seems to break things pretty well 2008-04-14 18:21 zcommits: [PPA zumastor-team] Accepted: ddsnap 0.8.0-r1535 (source) 2008-04-14 18:21 zcommits: [PPA zumastor-team] Accepted: zumastor 0.8.0-r1535 (source) 2008-04-14 18:21 zcommits: [zumastor commit] r1535 - trunk/zumastor/bin 2008-04-14 18:57 Later this week i'll write a test for the cluster to simulate whats going on upstairs 2008-04-14 18:58 (volumes on large md arrays with lvm) 2008-04-14 18:58 I can't tell if it is a raid/lvm/zumastor interaction issue or amd64 or what 2008-04-14 19:43 willn, and I'd appreciate a function trace 2008-04-14 19:43 kernel call trace I mean 2008-04-14 19:44 (another one) 2008-04-14 19:44 more call traces is better 2008-04-14 19:49 flips: Once I can get it on the test cluster, you can have all the debug info in the world :) 2008-04-14 19:49 willn: Even better is something along the lines of "this is the bug --->" ;-) 2008-04-14 19:49 willn, thx 2008-04-14 20:02 zcommits: [zumastor commit] r1536 - trunk/zumastor/bin 2008-04-14 20:03 Always fun when your two patches conflict with eachother 2008-04-14 20:13 -!- flips(~phillips@phunq.net) has joined #zumastor 2008-04-14 20:13 shapor, ping 2008-04-14 20:30 flips: He mentioned he would be unavail today 2008-04-14 20:35 right 2008-04-14 20:40 lemesee, how long does it take to copy one 500GB disk to another 2008-04-14 20:40 several long whiles 2008-04-14 20:41 -!- phoenix24(wrbj@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-14 20:42 4.6 hours I get, assuming the copy runs at 30 MB/sec 2008-04-14 20:42 I have yet to see a copy run at full bandwidth of the disks 2008-04-14 20:44 Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn 2008-04-14 20:44 sda 142.07 21955.99 4137.94 42166924 7946998 2008-04-14 20:44 sdb 28.28 3.26 21566.54 6252 41418962 2008-04-14 20:45 assuming it means block = 1k, that is only 22 MB/sec for the copy 2008-04-14 20:45 pathetic 2008-04-14 20:45 and meanwhile, system response is poor 2008-04-14 20:45 this is really not good at all 2008-04-14 20:46 this should run 3 times as fast and not perceptably impact interactive performance 2008-04-14 20:46 got low level badness in the block layer, other places 2008-04-14 20:46 poor io scheduler 2008-04-14 21:01 zissues: Issue 24 in zumastor: Snapshot squashing considered harmful 2008-04-14 21:01 actually, dd copies at 68 MB/sec 2008-04-14 21:01 so either cp is broken or iostat is broken 2008-04-14 21:01 the latter more likely 2008-04-14 21:08 -!- MaZe(~MaZe@c-24-6-86-168.hsd1.ca.comcast.net) has joined #zumastor 2008-04-14 21:09 wb maze 2008-04-14 21:12 zbot is very helpful 2008-04-14 21:13 -!- flips(~phillips@phunq.net) has left #zumastor 2008-04-14 21:13 -!- flips(~phillips@phunq.net) has joined #zumastor 2008-04-14 21:14 i should edit the pipe to provide some context 2008-04-14 21:14 as the mailing list isnt intelligent about showing issues multiple times 2008-04-14 21:17 it was nice just to know there was activity on the issue 2008-04-14 21:27 now iostat reports 70 mb/sec so I guess it is cp that is broken 2008-04-14 22:20 flips, pong 2008-04-14 22:34 shapr, hi 2008-04-14 22:53 -!- flips(~phillips@phunq.net) has joined #zumastor 2008-04-14 23:29 -!- phoenix24(jjp@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-14 23:32 cp -a of a 1 gig directory (mail archive) from one disk to another goes at 665 k/sec, vs 68 meg/sec for dd 2008-04-14 23:32 this is unspeakably horrible performance for the cp -a 2008-04-14 23:33 the filesystem is ext3 mounted with defaults 2008-04-14 23:44 patches welcome 2008-04-14 23:46 indeed 2008-04-14 23:46 physical readahead is likely the way to fix it 2008-04-14 23:47 could work on that instead of lvm3 ;) irc.oftc.net #zumastor log beginning Tue Apr 15 00:00:01 PDT 2008 2008-04-15 00:15 -!- erwan_taf(~erwan@LAubervilliers-151-13-63-69.w217-128.abo.wanadoo.fr) has joined #zumastor 2008-04-15 00:26 -!- flips(~phillips@phunq.net) has joined #zumastor 2008-04-15 00:26 my new 750 gig 5400 rpm dream disk is in 2008-04-15 00:27 so quiet 2008-04-15 00:27 and much cooler than the 500 gb 7200 rpm it replaces 2008-04-15 00:28 the hard disk is no longer the noisiest component of the system 2008-04-15 00:28 the quietized fan is 2008-04-15 00:28 the whole thing must be down around 28 db@1 meter 2008-04-15 00:28 somewhere between 26 and 27 db 2008-04-15 00:29 my sound meter isn't sensitive enough to measure it 2008-04-15 00:30 -!- phoenix24(ziqsk@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-15 01:01 -!- phoenix24(yrg@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-15 02:01 -!- phoenix24(tdgldkb@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-15 02:22 -!- erwan__taf(~erwan@LAubervilliers-151-13-63-69.w217-128.abo.wanadoo.fr) has joined #zumastor 2008-04-15 03:05 -!- phoenix24(wje@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-15 03:58 -!- pgquiles_(~pgquiles@222.Red-88-12-133.dynamicIP.rima-tde.net) has joined #zumastor 2008-04-15 04:07 -!- phoenix24(gvwenm@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-15 05:08 -!- pgquiles__(~pgquiles@62.43.226.52.static.user.ono.com) has joined #zumastor 2008-04-15 05:14 -!- phoenix24(ssk@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-15 05:18 zcommits: zumastor b0.8.0 r1535 build failure 255 2008-04-15 05:19 -!- pgquiles_(~pgquiles@222.Red-88-12-133.dynamicIP.rima-tde.net) has joined #zumastor 2008-04-15 05:25 -!- pgquiles__(~pgquiles@62.43.226.52.static.user.ono.com) has joined #zumastor 2008-04-15 05:32 -!- pgquiles__(~pgquiles@62.43.226.52.static.user.ono.com) has joined #zumastor 2008-04-15 05:33 zcommits: zumastor b0.8.0 r1536 build failure 1 2008-04-15 05:36 -!- pgquiles__(~pgquiles@222.Red-88-12-133.dynamicIP.rima-tde.net) has joined #zumastor 2008-04-15 05:43 -!- pgquiles_(~pgquiles@247.Red-79-152-213.staticIP.rima-tde.net) has joined #zumastor 2008-04-15 06:25 -!- phoenix24(garcyfs@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-15 06:47 -!- erwan_taf(~erwan@LAubervilliers-151-13-63-69.w217-128.abo.wanadoo.fr) has joined #zumastor 2008-04-15 10:18 Only a couple more issues to resolve for me, yay 2008-04-15 10:47 -!- MaZe(~MaZe@216-239-45-4.google.com) has joined #zumastor 2008-04-15 11:24 zcommits: [zumastor commit] r1537 - trunk/ddsnap 2008-04-15 11:28 -!- pgquiles(~pgquiles@138.Red-88-23-241.staticIP.rima-tde.net) has joined #zumastor 2008-04-15 11:50 -!- phoenix24(sfgw@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-15 11:59 I did a copy -a of a 70 GB directory and ended up with some files turning into directories on the destination 2008-04-15 11:59 also du is wrong by orders of magnitude 2008-04-15 12:00 stock 2.6.24.3 2008-04-15 12:00 ext3 2008-04-15 12:08 -!- phoenix24(tnjjq@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-15 12:39 -!- dkegel(~chatzilla@207.47.98.129.static.nextweb.net) has joined #zumastor 2008-04-15 12:59 flips: how'd you manage that 2008-04-15 12:59 good question 2008-04-15 13:00 ran gparted, it did some wierd things 2008-04-15 13:00 seemed to end up with a good partition table 2008-04-15 13:00 but the filesystem it created was braindamaged 2008-04-15 13:01 remade the filesystem, let's see if it continues stable 2008-04-15 13:03 So, looks like the AIO destroy/getevents race patch made it in to -mm! 2008-04-15 13:03 Probably won't make the mainstream until 2.6.26 (or a .25 maintanence release) though. 2008-04-15 13:03 kewl 2008-04-15 13:04 The general conclusion as to why this wasn't reported earlier is that nobody is doing io_destroy. They're just letting the context get wiped out on process exit. :-( 2008-04-15 13:04 Either that or they aren't using threads I guess. 2008-04-15 13:06 Thoughts on adding the patch to our list of kernel patches? 2008-04-15 13:08 -!- phoenix24_(ipkj@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-15 13:16 -!- MaZe(~MaZe@216-239-45-4.google.com) has joined #zumastor 2008-04-15 13:27 zissues: Issue 87 in zumastor: Clean up debian package build methodology 2008-04-15 13:28 cbsmith, send the patch to the list for review? 2008-04-15 13:28 dkegel: Already did. 2008-04-15 13:28 dkegel: You telling me it didn't show up? 2008-04-15 13:28 aha 2008-04-15 13:28 it's there 2008-04-15 13:28 dkegel: Okay, so this one didn't get marked as spam. Phew. 2008-04-15 13:29 Maybe you should file a zumastor issue and attach your patch there as the proposed fix... that'll make it more obvious you want it reviewed for inclusion in zumastor 2008-04-15 13:30 dkegel: Sure. That was the next step. I just wanted to get a feel from folks on IRC first. Me being timid and all. ;-) 2008-04-15 13:32 Jiaying's anxious to try out AIO... now that this hang is sorted, do you have any code for her to try? 2008-04-15 13:33 Aren't we all. :-) 2008-04-15 13:33 Yeah, I've got a preliminary one. Just need to apply this patch, back out our work around, and then test it a bit. 2008-04-15 13:42 zissues: Issue 111 in zumastor: Add kernel patch to resolve AIO destroy/getevents race. 2008-04-15 13:47 hooray for rss feeds and python irc bots 2008-04-15 13:47 and people who know how to write them 2008-04-15 13:49 I'm a big fan of pythons optparse library 2008-04-15 13:53 cbsmith, now get flips to review it :-) 2008-04-15 13:54 the aio hang bug patch? 2008-04-15 13:54 it looks fine 2008-04-15 13:55 flips: thx 2008-04-15 13:55 flips: I'll submit the patch to get it added to our kernel patches so it's all nice and tidy. 2008-04-15 13:55 cbsmith, why do we need to use io_destroy? 2008-04-15 13:56 flips: Well, in unit tests, it is a *really* good idea to clean up after yourself, if for no other reason than it makes it easier to track resource leaks. 2008-04-15 13:57 willn, you taught zbot to describe the issue I see 2008-04-15 14:10 -!- phoenix24(kqcgqv@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-15 14:15 Interesting LWN article on git bisect: http://lwn.net/Articles/277872/ 2008-04-15 14:36 wine has the same increasing reliance on bisect 2008-04-15 14:40 zcommits: [zumastor commit] r1538 - trunk/ddsnap/scripts 2008-04-15 14:46 zcommits: [PPA zumastor-team] Accepted: zumastor 0.8.0-r1538 (source) 2008-04-15 14:46 zcommits: [PPA zumastor-team] Accepted: ddsnap 0.8.0-r1538 (source) 2008-04-15 14:53 zissues: Issue 98 in zumastor: Move debian style kernel builds to the flavormaker tool 2008-04-15 15:16 zcommits: [zumastor commit] r1539 - in trunk/cbtb/tests: 1 2 3 2008-04-15 15:46 jiayingz: Looks like the 32/64bit compat fix worked 2008-04-15 15:46 Thanks! 2008-04-15 15:46 willn, thanks for the verification! 2008-04-15 16:18 I replicated 20G across the country to confirm :) 2008-04-15 16:24 hmm, seems like ddsnap delta listen can start > 1 time 2008-04-15 16:26 zcommits: [zumastor commit] r1540 - trunk/zumastor/patches/2.6.24.2 2008-04-15 16:26 Were we going to eliminate Tue Apr 15 19:23:54 2008: [11691] ddsnap_delta_server: unable to accept connection: Interrupted system call 2008-04-15 16:59 spam coming... 2008-04-15 16:59 * Unit tests, verify that: 2008-04-15 16:59 - first alloc in a new transaction is deferred 2008-04-15 16:59 - contiguous alloc extends deferred range 2008-04-15 16:59 - discontiguous alloc does not extend alloc range 2008-04-15 16:59 - on commit, defer is written into commit block 2008-04-15 16:59 - on commit, defer is added to defer list 2008-04-15 16:59 - on barrier, defer list is written to bitmap blocks 2008-04-15 16:59 - on barrier, bitmap blocks are flushed 2008-04-15 16:59 - on replay, allocs preceding a barrier are ignored 2008-04-15 16:59 - on replay, allocs following the last barrier are added to the defer list 2008-04-15 16:59 this is done in kind of a hacky unit test format at the moment 2008-04-15 16:59 better than it was though 2008-04-15 17:00 you have to code up a test wrapper (alternative main) and eyeball the output to ensure that things work as expected 2008-04-15 17:00 most probably will be much work to maintain 2008-04-15 17:01 but it sure picked up a bag of bugs 2008-04-15 17:09 Peaking 15MB/s for i386->amd64 replication 2008-04-15 17:09 tats not bad 2008-04-15 17:09 [with -g 6] 2008-04-15 17:10 what changed? 2008-04-15 17:12 Different disk confiuration 2008-04-15 17:18 Just about an hour for 10G replication 2008-04-15 17:18 (according to the bandwidth graph) 2008-04-15 17:26 I'm tempted to do a one-off openvz+zumastor package 2008-04-15 17:26 since i've got the automation now 2008-04-15 17:27 openvz has its own kernel? 2008-04-15 17:27 its got enough patches to build its own kernel 2008-04-15 17:34 [the patch they apply to the kernel is ~100k lines] 2008-04-15 17:57 ok, the bitmap optimization is passing lots of unit tests, just about time to take it for a test drive 2008-04-15 17:57 but first... need to skate 2008-04-15 18:48 -!- cbsmith(~user@adsl-99-165-23-73.dsl.lsan03.sbcglobal.net) has joined #zumastor 2008-04-15 19:22 zcommits: [zumastor commit] r1541 - trunk/kernel/ubuntu-package 2008-04-15 20:10 -!- flipz(~phlipz@phunq.net) has joined #zumastor 2008-04-15 20:17 what is nbock_write? 2008-04-15 20:19 is that the crude hack to avoid handling daemons the right way? 2008-04-15 21:08 yep 2008-04-15 22:22 zcommits: zumastor b0.8.0 r1537 build failure 255 2008-04-15 23:21 -!- phoenix24(pkoko@h1119083.serverkompetenz.net) has joined #zumastor irc.oftc.net #zumastor log beginning Wed Apr 16 00:00:01 PDT 2008 2008-04-16 00:43 -!- phoenix24(xswhbd@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-16 00:48 zcommits: zumastor b0.8.0 r1541 build failure 1 2008-04-16 01:32 -!- pgquiles_(~pgquiles@62.43.226.52.static.user.ono.com) has joined #zumastor 2008-04-16 01:35 -!- MaZe(~MaZe@c-24-6-86-168.hsd1.ca.comcast.net) has joined #zumastor 2008-04-16 01:43 -!- phoenix24_(irigxo@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-16 02:02 I managed to get the nsnaps ddsnap benchmark to run on my laptop 2008-04-16 02:02 and produce a gnuplot.ps even 2008-04-16 02:03 What is the nsnap benchmark ? 2008-04-16 02:03 nsnaps... the first and pretty much only benchmark we run 2008-04-16 02:03 successively untar a kernel tree, set a snapshot, repeat n times 2008-04-16 02:04 measure the time required for each untar 2008-04-16 02:04 ok 2008-04-16 02:04 it's about the worst case for ddsnap 2008-04-16 02:04 100% write once after setting a new snapshot 2008-04-16 02:05 I suppose the nice thing is, it's actually running without errors 2008-04-16 02:05 because it's running with the new optimized journal code 2008-04-16 02:05 I should try to convince myself it's actually running on the snapshot ;-) 2008-04-16 02:05 it must be 2008-04-16 02:06 otherwise ddnsnap snapshot would fail 2008-04-16 02:06 It must. 2008-04-16 02:06 I haven't looked into the test cases, let me do now. 2008-04-16 02:06 I'll be simplifying this one a little 2008-04-16 02:06 it wants a couple too many partitions to work with 2008-04-16 02:06 makes it inconvenient to run 2008-04-16 02:07 anyway 2008-04-16 02:07 it can be run by a sufficiently enthusiastic user ;-) 2008-04-16 02:07 first time I ever looked at it myself 2008-04-16 02:46 -!- phoenix24(xms@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-16 03:07 victory. 2008-04-16 03:08 the journalling optimization improved worst case performance from 383 seconds/untar to 342 seconds 2008-04-16 03:08 vs 41 for untar to raw volume 2008-04-16 03:09 that moves us from 9X to 8X performance 2008-04-16 03:09 worst case 2008-04-16 03:10 or 12.5% improvement 2008-04-16 03:16 This is significant 2008-04-16 03:17 yes, it corresponds pretty closely to the expected improvement 2008-04-16 03:17 there is another improvement of about the same size coming from a related technique 2008-04-16 03:18 then we have to switch to other techniques 2008-04-16 03:18 what is the other technique ? 2008-04-16 03:20 the current technque remembers which blocks were allocated by storing the info in the journal commit block instead of setting bitmaps 2008-04-16 03:20 this means the bitmaps don't have to be journalled 2008-04-16 03:20 the next technque does a similar thing with insersions into btree leaves 2008-04-16 03:21 instead of modifying the leaf, it will just remember in the journal commit block that an insertion was to be made 2008-04-16 03:21 and modify a whole bunch of leaves in a batch from time to time 2008-04-16 03:21 So, you are making the transactions more atomic ? 2008-04-16 03:22 hmm 2008-04-16 03:22 there isn't a term for it because it isn't a common technique ;-) 2008-04-16 03:22 hmm.. 2008-04-16 03:22 making the transactions smaller in terms of modified blocks 2008-04-16 03:23 by storing the tiny amount of information that actually must be remembered, in the journal commit block instead of updating some other block, which then has to be journalled to the disk 2008-04-16 03:23 yes 2008-04-16 03:25 as a side effect, you'r adding more consistency to snapshot. 2008-04-16 03:25 I think that in honor of this auspicious occasion, I should post the gnuplots for the benchmarks 2008-04-16 03:25 cool! 2008-04-16 03:26 no side effect, the consistency stays exactly the same 2008-04-16 03:26 updates to the bitmaps always were journalled 2008-04-16 03:26 now, instead of journalling whole bitmap blocks, we journal 16 bytes of info 2008-04-16 03:26 ah! ok 2008-04-16 03:27 which goes into the commit block, which has to be written anyway, so this is "for free" 2008-04-16 03:27 two block writes saved per write request serviced by the snapshot server 2008-04-16 03:29 yes 2008-04-16 03:30 daniel@moonbase:/src/zumadoc$ svn update 2008-04-16 03:30 svn: PROPFIND request failed on '/svn/www' 2008-04-16 03:30 svn: PROPFIND of '/svn/www': 502 Bad Gateway (https://zumastor.googlecode.com) 2008-04-16 03:30 ok, well I guess I have to put them on my own server then 2008-04-16 03:37 [Mon Apr 14 20:12:09 2008] [alert] mod_unique_id: unable to gethostbyname("moonbase.phunq.net") 2008-04-16 03:37 fsck 2008-04-16 03:37 I wonder why it's trying to look up that name 2008-04-16 03:44 http://moonbase.phunq.net/ddsnap/benchmarks/ <- can you see them? 2008-04-16 03:44 a min 2008-04-16 03:45 http://phunq.net/ddsnap/benchmarks/ <- this being the link ? 2008-04-16 03:46 yes 2008-04-16 03:47 -!- phoenix24(plqcifi@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-16 03:50 ddt-zumastor: sync bio.throttle.patch bugfix from svn 2008-04-16 03:50 ddt-zumastor: Fix fdsize64 return value 2008-04-16 03:50 ddt-zumastor: work in progress - cleanups and journal unit testing 2008-04-16 03:51 svn repo fails still 2008-04-16 03:51 :( 2008-04-16 03:52 google problem 2008-04-16 03:52 remember to complain to shapor when he gets up ;-) 2008-04-16 03:52 http://moonbase.phunq.net/ddsnap/benchmarks/optimized.nsnaps.jpg 2008-04-16 03:52 http://moonbase.phunq.net/ddsnap/benchmarks/baseline.nsnaps.jpg 2008-04-16 04:14 Is hibernating built into the linux-kernel ? or its a patchset ? 2008-04-16 04:18 got it 2008-04-16 04:35 -!- pgquiles__(~pgquiles@138.Red-88-23-241.staticIP.rima-tde.net) has joined #zumastor 2008-04-16 04:41 -!- pgquiles_(~pgquiles@62.43.226.52.static.user.ono.com) has joined #zumastor 2008-04-16 04:48 -!- phoenix24(lgvj@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-16 05:52 -!- phoenix24(udrhzio@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-16 05:53 -!- pgquiles__(~pgquiles@138.Red-88-23-241.staticIP.rima-tde.net) has joined #zumastor 2008-04-16 08:28 -!- pgquiles_(~pgquiles@62.43.226.52.static.user.ono.com) has joined #zumastor 2008-04-16 08:57 -!- mitchg(~mitchg@207.47.98.129.static.nextweb.net) has joined #zumastor 2008-04-16 09:56 -!- pgquiles_(~pgquiles@138.Red-88-23-241.staticIP.rima-tde.net) has joined #zumastor 2008-04-16 10:51 willn: ping 2008-04-16 10:54 willn: why are the gutsy zumastor and ddsnap packages not being built by launchpad? :-? (hardy's are) 2008-04-16 11:08 We had switched building to hardy since the 2.6.24 kernel is hardy only 2008-04-16 11:08 in theory id like to build for both 2008-04-16 11:12 hold that thought 2008-04-16 11:16 zcommits: [zumastor commit] r1542 - trunk 2008-04-16 11:17 Helps if I remember how if statements function... 2008-04-16 11:19 http://phunq.net/ddsnap/benchmarks/baseline.nsnaps.jpg looks better than http://phunq.net/ddsnap/benchmarks/optimized.nsnaps.jpg 2008-04-16 11:19 I wonder if Dan got them backwards 2008-04-16 11:21 zcommits: [PPA zumastor-team] Accepted: ddsnap 0.8.0-r1543 (source) 2008-04-16 11:21 zcommits: [PPA zumastor-team] Accepted: zumastor 0.8.0-r1543 (source) 2008-04-16 11:21 zcommits: [zumastor commit] r1543 - trunk 2008-04-16 11:21 Itd be swell if one could tell the PPA that the gutsy packages work fine on hardy 2008-04-16 11:27 gutsy packages available! :-) 2008-04-16 11:28 willn: I remember having read something about specifying more than one distribution in debian/changelog but when I tried that in my PPA it did not work (that was like 6 months ago, though) 2008-04-16 11:28 sadly I cannot remember how to do that 2008-04-16 11:29 oh yeah: http://www.debian.org/doc/debian-policy/ch-source.html 2008-04-16 11:29 4.4 2008-04-16 11:29 package (version) distribution(s); urgency=urgency 2008-04-16 11:31 If I upload once for each dist, it only takes the first 2008-04-16 11:31 either that or launchpad is broken temporarily 2008-04-16 11:31 zcommits: [zumastor commit] r1544 - trunk/cbtb/host-scripts 2008-04-16 11:33 in theory, "zumastor (0.6-r1543 gutsy hardy ; urgency=low)" should work 2008-04-16 11:33 and just one upload would be needed for both distributions 2008-04-16 11:34 zissues: Issue 95 in zumastor: support 64bit kernel 2008-04-16 11:35 fact is, that has never worked for me and I think it won't because in the 'pool' hierarchy there's a single tree for all the packages in a particular PPA, therefore there would be a filename clashing, ie. both the gutsy and the hardy package filename would be the same 2008-04-16 11:41 -!- phoenix24(nkx@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-16 11:41 -!- pgquiles_(~pgquiles@62.43.226.52.static.user.ono.com) has joined #zumastor 2008-04-16 11:42 mmm using zumastor as a rsync replacement is not working for me: initial replication works fine, the next one is never mounted 2008-04-16 11:42 root@dubna:~# zumastor status --usage 2008-04-16 11:42 VOLUME zumabooks: 2008-04-16 11:42 Status: running 2008-04-16 11:43 Snapshot store block size = 16384; 219 of 256 chunks free 2008-04-16 11:43 Origin size: 42,949,672,960 bytes 2008-04-16 11:43 Write density: 0 2008-04-16 11:43 Creation time: Wed Apr 16 17:34:27 2008 2008-04-16 11:43 Snap Creation time Usecnt Prio Chunks Unshared Shared 2008-04-16 11:43 2 Wed Apr 16 19:08:32 2008 0 0 ! ! ! 2008-04-16 11:43 3 Wed Apr 16 19:08:32 2008 0 0 ! ! ! 2008-04-16 11:43 totals 0 0 0 2008-04-16 11:43 that's with r1523 2008-04-16 11:43 and no 'define schedule' on the original machine, as I don't need old snapshots on that volume, just replication 2008-04-16 11:49 -!- pgquiles_(~pgquiles@138.Red-88-23-241.staticIP.rima-tde.net) has joined #zumastor 2008-04-16 11:56 -!- phoenix24(fkmfpc@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-16 12:00 oh, and a snapshot store of exactly 1 LVM extent (4MB in my case), which was the smallest I can make it 2008-04-16 12:01 Yea 2008-04-16 12:10 pgquiles_, how large is ur snapshot store 2008-04-16 12:11 looks like the snapshots got squashed 2008-04-16 12:11 that is why they were not mounted 2008-04-16 12:17 jiayingz: indeed, the logs said zumastor was trying to mount an already squashed snapshot 2008-04-16 12:17 jiayingz: so, what should be the snapshot store size if I only want replication? 2008-04-16 12:19 pgquiles, on downstream you still need a snapshot store that is large enough to hold all the replicated data in each cycle 2008-04-16 12:19 this depends on the workload 2008-04-16 12:20 and replication cycle 2008-04-16 12:20 how can I make an estimation? 2008-04-16 12:21 e.g., if you use 1 hour replication cycle, the snapshot store can be as small as write_bandwidth * 1 hour 2008-04-16 12:21 write bandwidth is the write throughput on upstream 2008-04-16 12:22 mmm so the longer the replication cycle, the bigger the snapshot store 2008-04-16 12:22 because that is maximum delta size sent to the downstream 2008-04-16 12:22 I was planning on a 1 day replication cycle :-/ 2008-04-16 12:23 then depends on if you are going to make a lot of writes in a day 2008-04-16 12:23 if the workload is always read dominant, a small snapshot store size can also work 2008-04-16 12:24 it's read dominant but sometimes we may write a huge amount of data (like 20 GB) in a day in one of the volumes 2008-04-16 12:24 but if you may overwrite the whole volume some time, it is safe to set snapshot store as large as the origin volume 2008-04-16 12:24 that constraint effectively renders zumastor useless as a rsync replacement :-( 2008-04-16 12:26 that's the cost of having atomic update - the data has to go somewhere until you flip the switch 2008-04-16 12:26 to be exactly a rsync replacement, we need to export origin volume instead of snapshot volume on downstream 2008-04-16 12:26 The question is: are you ok with your filesystem being inconsistent during update? If so, rsync is ok. 2008-04-16 12:26 dkegel, exactly 2008-04-16 12:29 pgquiles, i got an idea. now we have an option to limit replication bandwidth. 2008-04-16 12:30 if you limit the replication bandwidth to 1K. the maximum data that can be replicated a day is 86400K 2008-04-16 12:30 well, that is 86.4M 2008-04-16 12:31 problem is that may mean data is not replicated 2008-04-16 12:31 hmm, wait. that doesn't work. a replication cycle still needs to finish before the next one starts 2008-04-16 12:32 I want all my new data replicated but with a zero-sized (or at least really tiny) snapshot store 2008-04-16 12:33 but as dkegel said, the extra space is the cost you pay for holding old data before switching to the new version 2008-04-16 12:33 maybe the right way to do it is to introduce a new parameter to 'zumastor replicate', something like 'zumastor replicate --no-snapshots' which exports the origin volume instead of the snapshot volume to downstream 2008-04-16 12:34 you can use rsync for that purpose, but you may get inconsistent data during the replication 2008-04-16 12:34 I was trying to avoid having to set up both zumastor and rsync on the same server, zumastor for some volumes, rsync for others 2008-04-16 12:35 but if I have to choose between wasting that much space and setting up rsync, I'm going with rsync for those volumes 2008-04-16 12:35 pgquiles, are you ok with the volume being inconsistent during replication? That's the price of using rsync (or this hack Jiaying is proposing) 2008-04-16 12:36 what is rsync used for? just for backup? 2008-04-16 12:36 jiayingz: yes, just for having data replicated to several locations. We do not need snapshots for frozen software, or for third party software. 2008-04-16 12:37 Do you have some sort of policy in place, e.g. write-once, that keeps inconsistency from being a problem? 2008-04-16 12:37 dkegel: I'd like it better if data was always consistent on downstream but given that those servers are there just for backup, inconsistency does not worry me 2008-04-16 12:37 OK, then the option Jiaying proposes is worth looking at. Thanks for the nudge. 2008-04-16 12:39 dkegel: yes, we have something like that: we have several servers and we always write and read to one of them. If that server goes down, another server takes its place and users do not realize they are accessing a different server (they access it with the very same name the now-broken server had) 2008-04-16 12:39 pgquiles, for record, could you file an issue for this? 2008-04-16 12:39 jiayingz: sure 2008-04-16 12:41 issue 112 2008-04-16 12:48 pgquiles: stonith? 2008-04-16 12:57 -!- phoenix24(lxrz@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-16 12:57 willn: stonith? what? :-? 2008-04-16 12:57 willn: oh, no 2008-04-16 13:08 hey 2008-04-16 13:08 I'm issing a fun chat 2008-04-16 13:09 any chat involving stonith has got to be fun ;-) 2008-04-16 13:09 What is stonith ? 2008-04-16 13:09 http://www.linux-ha.org/STONITH 2008-04-16 13:09 zissues: Issue 112 in zumastor: Zumastor as a rsync replacement requires wasting of space 2008-04-16 13:10 most fun thing is two machines repeated stonithing each other 2008-04-16 13:10 one goes down, it comes up and shoots the other down 2008-04-16 13:10 happens often 2008-04-16 13:11 hmm, zbot has built in support for hg 2008-04-16 13:11 go figure 2008-04-16 13:11 zbot is a canned script? 2008-04-16 13:12 gozerbot 2008-04-16 13:14 python, figures 2008-04-16 13:14 in a good way 2008-04-16 13:14 tinyurl https://edge.launchpad.net/~zumastor-team/+archive/+builds?build_text=&build_state=all 2008-04-16 13:15 well, that failed 2008-04-16 13:15 ooh, is zbot supposed to make it so? 2008-04-16 13:15 zbot: tinyurl https://edge.launchpad.net/~zumastor-team/+archive/+builds?build_text=&build_state=all 2008-04-16 13:15 http://tinyurl.com/646e7r .. http://preview.tinyurl.com/646e7r 2008-04-16 13:15 there we go. 2008-04-16 13:15 that is a pretty site 2008-04-16 13:21 waiting for zbot to notice my git checkin 2008-04-16 13:21 http://phunq.net/ddtree?p=zumastor/.git;a=summary 2008-04-16 13:21 it used to notice those, didn't it? 2008-04-16 13:22 it does periodic rss pulls, since the feeds are large 2008-04-16 13:22 especially yours 2008-04-16 13:22 ddt-zumastor: Configure nsnaps test and reduce the number of partitions required for the 2008-04-16 13:23 speak of the zbot 2008-04-16 13:27 nice zbot 2008-04-16 13:31 pgquiles, still there? 2008-04-16 13:37 flips: I was almost in bed, but yes, I'm here 2008-04-16 13:38 re your rsync replacement comment, zumastor can be extended to cover the case you want 2008-04-16 13:38 that'd be great 2008-04-16 13:38 willn, looks like 'apt-get remove' still asks input even with '--force-yes' 2008-04-16 13:38 it is possible to fall back to an rsync-like method called rdiff that works on volumes instead of files 2008-04-16 13:39 you still do need to be able to maintain an upstream snapshot during the period of the rdiff 2008-04-16 13:39 if snapshot store fills up in that time and then the snapshot will have to be dropped and the rdiff restarted 2008-04-16 13:39 but it should be a nice fallback 2008-04-16 13:39 I'm not sure it would be really useful 2008-04-16 13:40 it still wastes space, therefore rsync may make more sense in that case 2008-04-16 13:40 otherwise, getting an exact copy of a volume while it is in use is impossible 2008-04-16 13:40 I understand 2008-04-16 13:40 rsync is good when you don't have that many files to check and when exact copy is not important 2008-04-16 13:40 rsync remains available of course 2008-04-16 13:41 on can also fall back to rsync of course 2008-04-16 13:41 I wonder if zumastor should try to support that idea 2008-04-16 13:41 something to think about 2008-04-16 13:41 that's the point in this case: if you don't want snapshots, you don't care the volume downstream is a block-by-block copy of the origin, you just want all the data gets transferred, even if it's not a point-in-time copy 2008-04-16 13:42 pgquiles, is it ok to have the snapshot store upstream at least? It only has to be big enough to hold the new directories in your use case... 2008-04-16 13:42 dkegel: no, it does not make sense upstream either 2008-04-16 13:43 think about it this way: you don't need a point-in-time snapshot, just the data copied over to downstream 2008-04-16 13:44 And you're ok with the downstream volume being offline while the data is being copied? 2008-04-16 13:44 if the replication was started at 22.00 and finished at 23.30, and at 22.05 someone updated in the origin a file you had already replicated, it's OK to have the old version downstream 2008-04-16 13:44 dkegel: that's even worse 2008-04-16 13:44 maybe for this use case the copy should be done at file-level, not at block-level 2008-04-16 13:44 i.e. what rsync does 2008-04-16 13:44 I'm trying to scare you off 2008-04-16 13:44 :-) 2008-04-16 13:44 :-D 2008-04-16 13:45 If rsync is working for you, keep using it. Zumastor is for people who are unhappy with rsync. 2008-04-16 13:45 jiayingz: in what context? 2008-04-16 13:46 willn, uml smoke test 2008-04-16 13:46 Which script? 2008-04-16 13:46 pgquiles, we just had a user who is using rsync to mirror five million files. It blew up. He's now only doing batches of 100K files. 2008-04-16 13:46 willn, all those that include 'apt-get remove' 2008-04-16 13:46 dkegel: it's not a question of rsync OR zumastor, it's more like having two tools for more or less the same work. For some volumes I only need replication, for others I want snapshots and replication, and replicated snapshots downstream, too 2008-04-16 13:47 Does the fact that disks are getting cheaper make snapshots a bit more appealing? 2008-04-16 13:47 jiayingz: oh right, e2fsprogs does that. 2008-04-16 13:47 dkegel, zumastor can have better replication performance than rsync with large volume size 2008-04-16 13:48 Yes, and that might translate into "not happy with rsync". Until it does, he should keep using rsync. 2008-04-16 13:48 willn, so that is not 'apt-get remove'. that is 'e2fsprogs' 2008-04-16 13:48 in my case, disks are not a problem: I've got plenty of disk space. Actually it's me trying to find if I could avoid installing and setting up both zumastor and rsync. Having some wasted disk is not actually a problem (at least currently :-) 2008-04-16 13:49 its complaining because e2fsprogs is a core part of ubuntu 2008-04-16 13:49 so when you remove it, you break major things 2008-04-16 13:49 I'll commit a fix 2008-04-16 13:49 willn, thx! 2008-04-16 13:52 r1545 2008-04-16 13:57 zcommits: [zumastor commit] r1545 - trunk/cbtb/tests/1 2008-04-16 13:57 zbot: botsnack 2008-04-16 13:57 smikkel ;] 2008-04-16 13:58 ok, now I'm really going to bed 2008-04-16 13:58 see you! 2008-04-16 14:01 flips, how is your optimization work going? 2008-04-16 14:02 jiayingz, debugging 2008-04-16 14:02 And are those graphs backwards? 2008-04-16 14:02 I mean, are the names reversed? 2008-04-16 14:02 there're right, there is a bug 2008-04-16 14:02 that produces the opposite effect that was intended 2008-04-16 14:02 tracking down now 2008-04-16 14:03 blow by blow is recorded here: http://phunq.net/ddtree?p=zumastor/.git;a=summary 2008-04-16 14:04 just need to keep working on it, it will crack 2008-04-16 14:04 improving the unit test now to run something more like the production load I tested, under debuggable conditions 2008-04-16 14:05 flips, have u stopped using svn totally? 2008-04-16 14:06 no 2008-04-16 14:06 for this extended hack it isn't appropriate though 2008-04-16 14:07 when the benchmarks are showing the expected numbers, then will come a phase of putting the patches back into svn 2008-04-16 14:07 no sense starting that until we've seen the expected numbers 2008-04-16 14:08 ...getting rid of really_init_snapstore right now 2008-04-16 14:08 making that sane 2008-04-16 14:10 flips, in ur fd_size, bytes is not really needed 2008-04-16 14:10 u may get compiling warnnings because bytes is a pointer 2008-04-16 14:10 it's already fixed 2008-04-16 14:11 it was very wrong before 2008-04-16 14:11 it's in the changelog 2008-04-16 14:16 ok, there goes another 180 line diff and it is necessary in order for initialization to be sane 2008-04-16 14:37 ddsnapd still starts, good 2008-04-16 14:37 really_init_snapstor is going to die very soon 2008-04-16 14:39 willn, i think u used 'zumastor forget xxx' instead of 'zumastor forget volume xxx' in ur cbtb tests 2008-04-16 14:39 shoot. 2008-04-16 14:42 Fixed in r1546 2008-04-16 14:42 Thanks for catching those :) 2008-04-16 14:43 np 2008-04-16 14:47 zcommits: [zumastor commit] r1546 - trunk/cbtb/tests/1 2008-04-16 15:04 -!- phoenix24(zgtbi@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-16 15:15 zissues: Issue 113 in zumastor: ddsnap auto_delete code skips zero-usecount squashed snapshot 2008-04-16 15:25 git-clone takes forever... 2008-04-16 16:12 -!- phoenix24(hueo@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-16 17:12 -!- cbsmith(~user@207.47.98.129.static.nextweb.net) has joined #zumastor 2008-04-16 17:13 Hmmm... this looks like a bug in get_status(): 2008-04-16 17:13 while (path[level].pnext < node->entries + node->count) { 2008-04-16 17:13 leafbuf = snapread(sb, path[level].pnext++->sector); 2008-04-16 17:13 if (!leafbuf) { 2008-04-16 17:13 warn("unable to read leaf at sector 0x%Lx of tree traversal", 2008-04-16 17:13 level? path[level - 1].pnext++->sector: sb->image.etree_root); 2008-04-16 17:13 2008-04-16 17:13 ick... let me use pastebin. 2008-04-16 17:15 http://pastebin.com/d2a0caa17 2008-04-16 17:15 So, it looks to me like the warning message is not going to be using the right value of pnext. 2008-04-16 17:15 Perhaps I'm grokking it wrong though. 2008-04-16 17:16 Why does it use the pnext from the level below current level? 2008-04-16 17:16 And why does it mutate it? 2008-04-16 17:31 blah, ubuntu openvz is broken 2008-04-16 17:31 first it doesnt boot, then you build from git and mv/cp don't work 2008-04-16 17:40 zissues: Issue 114 in zumastor: snapshot kept for multiple rotations is umounted when it is removed from one rotation 2008-04-16 17:42 willn: Try Gentoo openvz. Seems to work just fine for me. 2008-04-16 17:44 ehm. I'm not much of a gentoo guy 2008-04-16 17:53 -!- MaZe(~MaZe@216-239-45-4.google.com) has joined #zumastor 2008-04-16 19:31 funroll loops! 2008-04-16 20:22 procname for origin snapshot has now become really ugly, my fault I think: ddcli/ffffffff 2008-04-16 20:22 :P 2008-04-16 21:57 ACTION spots the bug in journal optimization 2008-04-16 21:58 backwards conditional in limit_deferred_allocs :-] 2008-04-16 22:05 -!- MaZe(~MaZe@c-24-6-86-168.hsd1.ca.comcast.net) has joined #zumastor 2008-04-16 22:14 zcommits: zumastor b0.8.0 r1542 build failure 127 2008-04-16 22:39 zcommits: zumastor b0.8.0 r1546 build failure 1 2008-04-16 22:52 woohoo 2008-04-16 22:52 I think openvz builds work 2008-04-16 23:04 zcommits: [zumastor commit] r1547 - trunk/openvz/package-builder 2008-04-16 23:15 ok, it's doing the right thing now (& creating a massive log to convince me) 2008-04-16 23:18 hm? 2008-04-16 23:18 the journalling optimization 2008-04-16 23:18 it's quite the elaborate hack 2008-04-16 23:18 patches will start landing in a few days I think, you'll see ;-) 2008-04-16 23:18 now to remove the logging and see if it actually optimizes this time 2008-04-16 23:19 the more often the better 2008-04-16 23:20 here goes the real test 2008-04-16 23:20 (nsnaps benchmark) 2008-04-16 23:21 running on a laptop that doesn't have enough memory go guarantee no deadlock 2008-04-16 23:21 got to set up a better test machine 2008-04-16 23:27 ok, it's running faster than baseline 2008-04-16 23:27 but I forgot to take out a line of log info 2008-04-16 23:28 here we go 2008-04-16 23:29 first attempt only saved 2% 2008-04-16 23:29 hopefully the log output was the reason for not more 2008-04-16 23:40 well there we have it, the last three-four weeks work saved 1.7% of runtime :p 2008-04-16 23:40 it did optimize 2008-04-16 23:41 it's just that we have far worse problems than the 5 metadata writes per write request, it is now apparent 2008-04-16 23:41 do need to compute stats to find out whether the expected number of metadata writes were saved 2008-04-16 23:42 if that checks out, then clearly we need to be working on other aspects of optimization 2008-04-16 23:54 ddt-zumastor: refactor ddsnap initialization and refactor this optimization to actually work irc.oftc.net #zumastor log beginning Thu Apr 17 00:00:01 PDT 2008 2008-04-17 00:02 From the MySQL User's Conference, Sun has announced, and former CEO Marten Mickos has confirmed, that Sun will be close sourcing sections of the MySQL code base. 2008-04-17 00:07 shapor, I'm on top of it... http://developers.slashdot.org/comments.pl?sid=525246&cid=23101092 2008-04-17 00:08 ok, that's enough hacking for today 2008-04-17 00:08 I'm PO'd about the bottom line result, only because spending effort here took time away from optimizations that matter more 2008-04-17 00:09 on the other hand it was fun 2008-04-17 00:09 still is 2008-04-17 00:10 1% worst case speedup multiplied by thousands of zumastor machines will recover the development time in... oh... about 80 years? 2008-04-17 00:10 ACTION is probably being a little pessimistic there 2008-04-17 00:10 you mean at least 1% speedup? 2008-04-17 00:11 what sense of "at least"? 2008-04-17 00:11 oh i see your post now 2008-04-17 00:12 actually, there's no question about the value of the work, even forgetting about the optimization, there's a shiny new git repo that works wonders for my development and testing 2008-04-17 00:12 are you sure you've reduced the ios? 2008-04-17 00:12 and there are a _lot_ of badly needed cleanups 2008-04-17 00:12 like watching iostat 1 2008-04-17 00:12 I must have reduced them otherwise there would have been no speedup 2008-04-17 00:13 the journal might be too small 2008-04-17 00:13 I'm probably going to find something like that 2008-04-17 00:13 it should be apparent in the origin/snapshot io ratio 2008-04-17 00:13 if you actualy did reduce the number of writes 2008-04-17 00:13 I will add stats to the log output, maybe a timer to dump there every now and then, and know much more 2008-04-17 00:13 where do we see that ratio? 2008-04-17 00:14 you mean, that's a stat I should print out? 2008-04-17 00:14 and easy thing to do is print out the stats on client disconnect 2008-04-17 00:14 we might as well always do that 2008-04-17 00:15 tomorrow's hack 2008-04-17 00:15 i usually just get that from watching iostat 2008-04-17 00:15 ah, should have ;-) 2008-04-17 00:15 from your posted result it looks like it slowed down? 2008-04-17 00:15 oops 2008-04-17 00:15 I wrote it wrong then 2008-04-17 00:15 :) 2008-04-17 00:15 fsck 2008-04-17 00:17 there we go 2008-04-17 00:36 -!- pgquiles(~pgquiles@138.Red-88-23-241.staticIP.rima-tde.net) has joined #zumastor 2008-04-17 00:46 ...my diff against the fork point 3 weeks ago is 5581 lines :-/ 2008-04-17 00:47 ah, that includes a bogus ddsnap.c.orig file that crept in there 2008-04-17 00:49 so the true stats are: 13 files changed, 1032 insertions(+), 617 deletions(-) 2008-04-17 00:49 and total diff is 2430 lines 2008-04-17 00:49 still a pretty serious hack by volume alone 2008-04-17 01:16 -!- erwan_taf(~erwan@81.80.43.67) has joined #zumastor 2008-04-17 01:28 -!- phoenix24(jbs@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-17 01:30 -!- pgquiles_(~pgquiles@62.43.226.52.static.user.ono.com) has joined #zumastor 2008-04-17 02:19 -!- pgquiles__(~pgquiles@138.Red-88-23-241.staticIP.rima-tde.net) has joined #zumastor 2008-04-17 02:29 -!- phoenix24(rhyhn@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-17 02:53 -!- phoenix24(ouqbqwz@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-17 03:30 -!- pgquiles_(~pgquiles@62.43.226.52.static.user.ono.com) has joined #zumastor 2008-04-17 03:37 -!- pgquiles__(~pgquiles@138.Red-88-23-241.staticIP.rima-tde.net) has joined #zumastor 2008-04-17 03:49 -!- pgquiles_(~pgquiles@62.43.226.52.static.user.ono.com) has joined #zumastor 2008-04-17 04:07 -!- phoenix24(aeimuu@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-17 04:17 -!- pgquiles__(~pgquiles@138.Red-88-23-241.staticIP.rima-tde.net) has joined #zumastor 2008-04-17 04:48 -!- pgquiles_(~pgquiles@62.43.226.52.static.user.ono.com) has joined #zumastor 2008-04-17 05:06 -!- pgquiles__(~pgquiles@138.Red-88-23-241.staticIP.rima-tde.net) has joined #zumastor 2008-04-17 05:15 -!- phoenix24(xfjgalm@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-17 07:30 -!- pgquiles__(~pgquiles@138.Red-88-23-241.staticIP.rima-tde.net) has joined #zumastor 2008-04-17 08:15 -!- pgquiles_(~pgquiles@62.43.226.52.static.user.ono.com) has joined #zumastor 2008-04-17 08:22 how do I tell zumastor to forget a schedule? Initially I used "zumastor define schedule zumahistoric --monthly 30", then I realized what I actually wanted was "zumastor define schedule zumahistoric --daily 30" and now I have both schedules and no "zumastor forget schedule ... " command :-? 2008-04-17 08:31 -!- pgquiles__(~pgquiles@138.Red-88-23-241.staticIP.rima-tde.net) has joined #zumastor 2008-04-17 08:38 zumastor define schedule zumahistoric --monthly 0 --daily 0 2008-04-17 08:38 maybe? 2008-04-17 08:39 it would make sense for forget schedule to work thogh 2008-04-17 08:42 shapor: "--monthly 0" makes sense, I'll try that later 2008-04-17 08:42 I'm putting zumastor into production right now with 5 volumes totaling 1.5TB 2008-04-17 08:43 :-) 2008-04-17 08:43 sweet 2008-04-17 08:43 only two servers currently, 2 more to come next week after I see everything's working fine with these two 2008-04-17 08:44 also, I forgot to tell zumastor not to compress deltas, is it possible to redefine that now? 2008-04-17 08:45 maybe a "zumastor set property value" would make sense 2008-04-17 08:58 what version are you running, i dont think the ability to not compress is in 0.7 2008-04-17 09:01 I'm running trunk 2008-04-17 09:01 you should be able to re-run "define target" 2008-04-17 09:02 with -u 2008-04-17 09:02 it should just update the config 2008-04-17 09:03 ok 2008-04-17 09:38 -!- pgquiles_(~pgquiles@62.43.226.52.static.user.ono.com) has joined #zumastor 2008-04-17 09:54 -!- phoenix24(uoc@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-17 09:54 Hi !! 2008-04-17 09:54 -!- pgquiles__(~pgquiles@138.Red-88-23-241.staticIP.rima-tde.net) has joined #zumastor 2008-04-17 10:21 -!- jiayingz(~jiayingz@207.47.98.129.static.nextweb.net) has joined #zumastor 2008-04-17 10:38 zcommits: [zumastor commit] r1548 - trunk/openvz/package-builder 2008-04-17 10:58 -!- phoenix24(xrmei@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-17 11:31 ACTION heads in 2008-04-17 12:01 -!- phoenix24(yjbyiv@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-17 12:02 pgquiles_, rather than a "set property" approach, I favor a "zumastor redefine" command, which changes any properties specified and leaves properties not specified as they were 2008-04-17 12:02 how are those two approaches different? 2008-04-17 12:03 pgquiles_, in other words, how "zumastor define" works now, and "zumastor define" would be changed to replace all properties not mentioned to defaults 2008-04-17 12:04 willn, the syntax of "set property" gets problematic since properties can be global, per-volume, per-master, etc 2008-04-17 12:04 but we already have a command "zumastor define" that knows how to apply settings 2008-04-17 12:04 ah. 2008-04-17 12:04 I pity the one making changes to our option parsing 2008-04-17 12:05 compared to the hack I just did it's easy ;-) 2008-04-17 12:05 let's go have a look, see if it got more compex recently... 2008-04-17 12:05 its hideous 2008-04-17 12:06 who shall be blame, shapor? 2008-04-17 12:06 or me? 2008-04-17 12:06 I won't point fingers 2008-04-17 12:06 a discreet nod in the direction of the culprit then 2008-04-17 12:07 see, bash isn't too good at abstracting 2008-04-17 12:07 s/abstracting/anything/ 2008-04-17 12:07 we're pretty much stuck with the "getopt" followed by case from 2008-04-17 12:07 form 2008-04-17 12:11 willn, which command has the worst options parsing? 2008-04-17 12:12 Their all pretty gross. Some have multiline usage with -h, some have single line usage 2008-04-17 12:12 or --help now 2008-04-17 12:12 let's kill the -h's 2008-04-17 12:12 their gone 2008-04-17 12:13 That was the bug I assigned to you, but fixed myself 2008-04-17 12:13 have you looked at the option parsing in ddsnap.c? then you might not complain about the stuff in zumastor ;-) 2008-04-17 12:13 I'm avoiding ugly code 2008-04-17 12:14 and hey, you wrote it 2008-04-17 12:14 feast your eyes 2008-04-17 12:14 only the bash parsing 2008-04-17 12:14 the ddsnap.c stuff was other artists 2008-04-17 12:15 I don't really see a problem with the zumastor parsing 2008-04-17 12:15 you have to accept the beyond-ugly way that getopt works in bash 2008-04-17 12:15 otherwise it is pretty straightforward, easy enough to read and change 2008-04-17 12:16 just not possible to abstract, like writing a generic help output for example 2008-04-17 12:16 flips: do you mean there would be both a "zumastor define" command and a "zumastor redefine" command? 2008-04-17 12:16 pgquiles__, that is my proposal 2008-04-17 12:16 flips: ok, fine 2008-04-17 12:17 currently we effectively have only the "redefine" form 2008-04-17 12:17 that is useful, but surprising 2008-04-17 12:18 one more request: add a "--offset" parameter to zumastor replicate and zumastor define schedule, to be able to define what time the snapshot/replication is started 2008-04-17 12:18 for instance if define schedule is --daily, --offset should take hh:mm 2008-04-17 12:18 if --monthly, --offset should take dd:hh:mm 2008-04-17 12:19 ACTION points the finger at flips for option parsing :) 2008-04-17 12:19 --hourly => --offset mm 2008-04-17 12:19 :-] 2008-04-17 12:19 pgquiles__ yeah i believe we had a request for offset before 2008-04-17 12:20 netapp has some 4,5,6@30 syntax iirc 2008-04-17 12:20 we can technically support that now, you just have to setup your own cron job ;)\ 2008-04-17 12:20 or modify the existing one 2008-04-17 12:21 i think while zumasotr is bash we were going to defer that 2008-04-17 12:21 I'm going to check my python script into the git repo and start maintaining it I think 2008-04-17 12:21 it will be useful for test-driving ddsnap at least 2008-04-17 12:21 -!- MaZe(~MaZe@216-239-45-4.google.com) has joined #zumastor 2008-04-17 13:02 -!- phoenix24(gjlfp@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-17 13:14 slow day for commits 2008-04-17 14:04 -!- pgquiles_(~pgquiles@62.43.226.52.static.user.ono.com) has joined #zumastor 2008-04-17 14:13 what does "Status: failed" mean in the output of "zumastor status --usage"? Volumes seem to replicate correctly :-? 2008-04-17 14:19 zcommits: zumastor b0.8.0 r1547 build success 2008-04-17 14:29 zcommits: zumastor b0.8.0 r1548 build success 2008-04-17 14:29 zcommits: zumastor b0.8.0 r1547 install failure 1 2008-04-17 14:39 zcommits: zumastor b0.8.0 r1548 install failure 1 2008-04-17 14:42 -!- pgquiles__(~pgquiles@138.Red-88-23-241.staticIP.rima-tde.net) has joined #zumastor 2008-04-17 15:21 pgquiles_, was 'ddsnap server' running when you saw that 'zumastor status' error? 2008-04-17 15:28 -!- phoenix24(ybye@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-17 15:34 jiayingz: now it says "Status: running", I guess ddsnap_server was not running when status was 'failed' but I wonder why, because a replication was going on :-? 2008-04-17 15:35 replication can be going on without ddsnap server 2008-04-17 15:35 replication only needs to talk to 'ddsnap server' at the beginning 2008-04-17 15:36 but it is strange why 'ddsnap server' was not there 2008-04-17 15:36 is there any information form server.log? 2008-04-17 15:39 jiayingz: no, everything looks fine in the logs, for all the volumes 2008-04-17 15:40 I'll double my attention to these servers for the next few days just in case something odd is going on 2008-04-17 15:41 pgquiles, let us know if you see the problem again 2008-04-17 15:41 btw, is it possible to re-define a target while a replication cycle is going on :-? 2008-04-17 15:41 (I want to add -u to that target) 2008-04-17 15:41 what is -u? 2008-04-17 15:41 send data uncompressed 2008-04-17 15:42 s/data/deltas 2008-04-17 15:43 ah, right, the feature will added 2008-04-17 15:43 that may not be a good idea 2008-04-17 15:43 that's what I thought :-) 2008-04-17 15:43 a safe way wil be echoing 1 to that file 2008-04-17 15:44 :-? 2008-04-17 15:44 what do you mean? to what file? 2008-04-17 15:44 sorry, echo 0 to /var/lib/zumastor/$vol/targets/$host/compress 2008-04-17 15:44 ah, ok 2008-04-17 15:44 or just remove that file 2008-04-17 15:44 are deltas uncompressed as they are received downstream, or at the end of the replication cycle? 2008-04-17 15:44 that is the safe but hacky way :) 2008-04-17 15:46 deltas are uncompressed as they are received 2008-04-17 15:46 ok 2008-04-17 15:46 one of the volumes I'm replicating is taking a long time due to compression 2008-04-17 15:47 how do u know it is compression? 2008-04-17 15:51 -!- pgquiles_(~pgquiles@62.43.226.52.static.user.ono.com) has joined #zumastor 2008-04-17 15:51 [00:48] I assume that's the problem because two other volumes with more or less the same amount of data took way shorter to replicate 2008-04-17 15:51 [00:49] and because when I transferred the files from the original server to this soon-to-be-the-real-server using rsync -avz it was taking ages 2008-04-17 15:51 [00:49] I stopped it, tried again with rsync -av and it finished within reasonable time 2008-04-17 15:51 [00:50] oh, great, I'm disconnected but Konversation has not realized yet :-/ 2008-04-17 15:51 VPNing to work makes Konversation go mad :-D 2008-04-17 15:52 Did we unlist the snapshots directory from zumastor.org 2008-04-17 15:53 i c 2008-04-17 15:54 willn, what do u mean 'unlist the snapshots directory'? 2008-04-17 15:54 I see how that was confusing :) -- http://zumastor.org/downloads/snapshots/ hasn't been updated in a while 2008-04-17 16:01 -!- pgquiles__(~pgquiles@138.Red-88-23-241.staticIP.rima-tde.net) has joined #zumastor 2008-04-17 16:18 is * replication checksum error when volume size is not a multiple or chunk size 2008-04-17 16:18 still an issue? 2008-04-17 16:18 i thought that was closed long time ago 2008-04-17 16:19 ok 2008-04-17 16:30 zcommits: [zumastor commit] r1549 - in trunk: . doc 2008-04-17 16:35 zcommits: [zumastor commit] r1550 - trunk/cbtb/host-scripts 2008-04-17 16:39 -!- phoenix24(ymjto@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-17 16:40 zcommits: zumastor b0.8.0 r1550 build success 2008-04-17 16:44 install? 2008-04-17 16:48 so, nblock_write is the only component in zumastor causing it to be built constantly 2008-04-17 16:49 where built is compliled instead of packaged for any 2008-04-17 16:50 zcommits: zumastor b0.8.0 r1550 install success 2008-04-17 16:51 woo-hoo 2008-04-17 16:51 what does zumastor.conf get used for? 2008-04-17 16:54 blank for me 2008-04-17 16:54 me too. 2008-04-17 16:55 but, if the guesses are correct it must have obvious uses. 2008-04-17 16:55 nope. 2008-04-17 16:55 doesn't look like its referenced anywhere 2008-04-17 16:55 ah! not yet. 2008-04-17 18:01 -!- phoenix24(caftqih@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-17 19:02 -!- phoenix24(ugvw@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-17 19:42 -!- MaZe(~MaZe@216-239-45-4.google.com) has joined #zumastor 2008-04-17 20:40 zcommits: zumastor b0.8.0 r1550 test failure 65 2008-04-17 21:12 hda/sda again 2008-04-17 21:21 zcommits: [zumastor commit] r1551 - trunk/zumastor/etc/zumastor 2008-04-17 21:54 -!- phoenix24(cqizo@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-17 21:57 zissues: Issue 115 in zumastor: Turn zumastor into an arch-indep package 2008-04-17 22:06 zcommits: [zumastor commit] r1552 - in trunk: . ddsnap ddsnap/debian zumastor zumastor/debian zumastor/lib/zumastor zumastor/man 2008-04-17 22:06 Well, hopefully that wont require rollback 2008-04-17 22:11 zcommits: [PPA zumastor-team] Accepted: ddsnap 0.8.0-r1552 (source) 2008-04-17 22:15 it built 2008-04-17 22:15 thats a plus 2008-04-17 22:16 zcommits: [PPA zumastor-team] Accepted: zumastor 0.8.0-r1552 (source) 2008-04-17 22:21 zcommits: zumastor b0.8.0 r1552 build failure 1 2008-04-17 22:22 well thats not good 2008-04-17 22:24 ah, ok 2008-04-17 22:25 its because the builder is expecting to find a zumastor_version-revision_arch.deb 2008-04-17 22:25 instead of _all.deb 2008-04-17 22:41 zcommits: [zumastor commit] r1553 - trunk/cbtb/uml 2008-04-17 22:46 doing some testing... 2008-04-17 22:46 zcommits: [PPA zumastor-team] Accepted: ddsnap 0.8.0-r1553 (source) 2008-04-17 22:46 zcommits: [PPA zumastor-team] Accepted: zumastor 0.8.0-r1553 (source) 2008-04-17 22:52 zcommits: zumastor b0.8.0 r1553 build failure 1 2008-04-17 22:54 -!- phoenix24_(ich@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-17 22:57 zcommits: [zumastor commit] r1554 - trunk/cbtb/host-scripts 2008-04-17 22:59 ack 2008-04-17 23:00 this is getting ugly 2008-04-17 23:02 zcommits: [PPA zumastor-team] Accepted: ddsnap 0.8.0-r1555 (source) 2008-04-17 23:02 zcommits: [PPA zumastor-team] Accepted: zumastor 0.8.0-r1555 (source) 2008-04-17 23:02 zcommits: [zumastor commit] r1555 - trunk/cbtb/host-scripts 2008-04-17 23:06 well, this is complicated 2008-04-17 23:07 zcommits: zumastor b0.8.0 r1554 build failure 1 2008-04-17 23:12 zcommits: zumastor b0.8.0 r1555 build failure 1 2008-04-17 23:12 oh come on 2008-04-17 23:16 I guess it will be broken until the builder gets updated 2008-04-17 23:55 -!- phoenix24(tbtfidu@h1119083.serverkompetenz.net) has joined #zumastor irc.oftc.net #zumastor log beginning Fri Apr 18 00:00:01 PDT 2008 2008-04-18 00:38 -!- MaZe(~MaZe@c-24-6-86-168.hsd1.ca.comcast.net) has joined #zumastor 2008-04-18 00:55 -!- phoenix24_(upwqq@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-18 01:30 -!- pgquiles_(~pgquiles@62.43.226.52.static.user.ono.com) has joined #zumastor 2008-04-18 01:41 -!- pgquiles__(~pgquiles@138.Red-88-23-241.staticIP.rima-tde.net) has joined #zumastor 2008-04-18 02:03 -!- phoenix24(nxmkw@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-18 03:03 -!- phoenix24_(dpisxk@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-18 03:38 -!- pgquiles_(~pgquiles@62.43.226.52.static.user.ono.com) has joined #zumastor 2008-04-18 04:03 -!- phoenix24(tchvzx@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-18 04:05 -!- pgquiles__(~pgquiles@138.Red-88-23-241.staticIP.rima-tde.net) has joined #zumastor 2008-04-18 07:38 -!- pgquiles_(~pgquiles@62.43.226.52.static.user.ono.com) has joined #zumastor 2008-04-18 07:55 -!- pgquiles__(~pgquiles@138.Red-88-23-241.staticIP.rima-tde.net) has joined #zumastor 2008-04-18 09:23 -!- pgquiles_(~pgquiles@62.43.226.52.static.user.ono.com) has joined #zumastor 2008-04-18 10:13 ACTION blinks with fixes 2008-04-18 10:15 zcommits: [zumastor commit] r1556 - trunk/cbtb/host-scripts 2008-04-18 10:20 zcommits: [PPA zumastor-team] Accepted: zumastor 0.8.0-r1556 (source) 2008-04-18 10:20 zcommits: [PPA zumastor-team] Accepted: ddsnap 0.8.0-r1556 (source) 2008-04-18 10:25 zcommits: zumastor b0.8.0 r1556 build success 2008-04-18 10:25 zcommits: [zumastor commit] r1557 - trunk/cbtb/host-scripts 2008-04-18 10:30 zcommits: [PPA zumastor-team] Accepted: ddsnap 0.8.0-r1557 (source) 2008-04-18 10:35 zcommits: zumastor b0.8.0 r1556 install success 2008-04-18 10:35 zcommits: zumastor b0.8.0 r1557 build success 2008-04-18 10:45 zcommits: zumastor b0.8.0 r1557 install success 2008-04-18 10:46 woo! 2008-04-18 10:46 -!- pgquiles__(~pgquiles@138.Red-88-23-241.staticIP.rima-tde.net) has joined #zumastor 2008-04-18 10:55 zcommits: [PPA zumastor-team] Accepted: zumastor 0.8.0-r1557 (source) 2008-04-18 11:39 -!- MaZe(~MaZe@216-239-45-4.google.com) has joined #zumastor 2008-04-18 11:58 willn, woot! 2008-04-18 12:05 hi flips 2008-04-18 12:05 hi jiayingz 2008-04-18 12:05 have u thought about using dancing tree in ddsnap? 2008-04-18 12:05 seeing as I invented it... yes 2008-04-18 12:05 that is used in reiser4 2008-04-18 12:05 however, Netapp waved patents at me about it 2008-04-18 12:05 well 2008-04-18 12:06 that involved being part of a filesystem 2008-04-18 12:06 yes. 2008-04-18 12:09 -!- phoenix24(ejv@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-18 12:21 There was some other change I wanted to make re: what we hallway conferenced yesterday 2008-04-18 12:21 But I've forgotten now. 2008-04-18 13:28 -!- phoenix24(ato@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-18 13:33 -!- mitchg(~mitchg@207.47.98.129.static.nextweb.net) has joined #zumastor 2008-04-18 13:55 even with speedy disks in the xw6000 it takes forever and a year to setup a build 2008-04-18 14:01 zcommits: [zumastor commit] r1558 - trunk/zumastor/man 2008-04-18 14:11 zcommits: zumastor b0.8.0 r1558 build success 2008-04-18 14:16 zcommits: [zumastor commit] r1559 - trunk/cbtb/host-scripts 2008-04-18 14:26 zcommits: zumastor b0.8.0 r1559 build success 2008-04-18 14:26 zcommits: zumastor b0.8.0 r1558 install success 2008-04-18 14:31 -!- phoenix24(zjn@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-18 14:32 zcommits: zumastor b0.8.0 r1556 test failure 65 2008-04-18 14:32 zcommits: [zumastor commit] r1560 - in trunk/zumastor: etc/cron.daily etc/cron.hourly etc/cron.weekly lib/zumastor 2008-04-18 14:37 zcommits: zumastor b0.8.0 r1559 install success 2008-04-18 14:42 zcommits: zumastor b0.8.0 r1560 build success 2008-04-18 14:47 zcommits: zumastor b0.8.0 r1560 install success 2008-04-18 15:14 -!- MaZe(~MaZe@216-239-45-4.google.com) has joined #zumastor 2008-04-18 15:27 zcommits: [zumastor commit] r1561 - in trunk: ddsnap ddsnap/debian ddsnap/man zumastor zumastor/debian zumastor/scripts 2008-04-18 15:32 zcommits: zumastor b0.8.0 r1561 build success 2008-04-18 15:32 zcommits: [PPA zumastor-team] Accepted: zumastor 0.8.0-r1561 (source) 2008-04-18 15:32 zcommits: [PPA zumastor-team] Accepted: ddsnap 0.8.0-r1561 (source) 2008-04-18 15:34 zissues: Issue 117 in zumastor: Resolve ddsnap and zumastor lintian violations 2008-04-18 15:34 zissues: Issue 116 in zumastor: Fill out stub manpages 2008-04-18 15:47 zcommits: zumastor b0.8.0 r1561 install success 2008-04-18 16:13 zcommits: zumastor b0.8.0 r1559 test success 2008-04-18 16:34 zissues: Issue 118 in zumastor: Compiler warnings in ddsnap as of 1561 2008-04-18 17:48 zcommits: zumastor b0.8.0 r1561 test success 2008-04-18 20:36 -!- MaZe(~MaZe@216-239-45-4.google.com) has joined #zumastor 2008-04-18 21:58 -!- MaZe(~MaZe@216-239-45-4.google.com) has joined #zumastor 2008-04-18 22:05 -!- mlankhorst(~m@fw1.astro.rug.nl) has joined #zumastor 2008-04-18 22:57 -!- MaZe(~MaZe@c-24-6-86-168.hsd1.ca.comcast.net) has joined #zumastor 2008-04-18 23:21 -!- phoenix24(dqm@h1119083.serverkompetenz.net) has joined #zumastor irc.oftc.net #zumastor log beginning Sat Apr 19 00:00:01 PDT 2008 2008-04-19 01:34 -!- phoenix24(eqvvbbc@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-19 01:45 flips, ping 2008-04-19 02:40 -!- phoenix24(rscr@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-19 03:32 phoenix24, hi 2008-04-19 03:42 -!- pgquiles(~pgquiles@138.Red-88-23-241.staticIP.rima-tde.net) has joined #zumastor 2008-04-19 04:14 -!- phoenix24(czwh@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-19 05:38 -!- phoenix24(fcmqm@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-19 09:16 -!- pgquiles(~pgquiles@147.Red-88-18-198.staticIP.rima-tde.net) has joined #zumastor 2008-04-19 09:24 -!- phoenix24(wwgbx@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-19 09:30 -!- pgquiles_(~pgquiles@147.Red-88-18-198.staticIP.rima-tde.net) has joined #zumastor 2008-04-19 09:45 willn, there? 2008-04-19 09:57 -!- pgquiles__(~pgquiles@147.Red-88-18-198.staticIP.rima-tde.net) has joined #zumastor 2008-04-19 10:11 -!- pgquiles_(~pgquiles@238.Red-79-144-195.staticIP.rima-tde.net) has joined #zumastor 2008-04-19 14:30 -!- phoenix24(rzuyc@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-19 14:50 -!- phoenix24(clavohq@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-19 16:42 -!- pgquiles(~pgquiles@181.Red-81-34-4.dynamicIP.rima-tde.net) has joined #zumastor 2008-04-19 16:45 -!- pgquiles(~pgquiles@181.Red-81-34-4.dynamicIP.rima-tde.net) has joined #zumastor 2008-04-19 17:29 -!- phoenix24(cmxqhqj@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-19 17:35 -!- pgquiles_(~pgquiles@62.43.226.52.static.user.ono.com) has joined #zumastor 2008-04-19 17:42 -!- pgquiles__(~pgquiles@181.Red-81-34-4.dynamicIP.rima-tde.net) has joined #zumastor 2008-04-19 19:01 -!- phoenix24(yygz@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-19 19:32 hmm 2008-04-19 20:04 -!- phoenix24(pufy@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-19 21:18 -!- phoenix24(scvmfp@h1119083.serverkompetenz.net) has joined #zumastor irc.oftc.net #zumastor log beginning Sun Apr 20 00:00:01 PDT 2008 2008-04-20 00:29 -!- phoenix24(mzruuct@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-20 07:56 -!- phoenix24(xhgabi@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-20 09:36 zissues: Pipes Output 2008-04-20 09:54 hmm. 2008-04-20 10:30 -!- phoenix24(swnafzo@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-20 11:48 -!- phoenix24(ten@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-20 12:57 -!- pgquiles(~pgquiles@209.Red-88-18-198.staticIP.rima-tde.net) has joined #zumastor 2008-04-20 12:58 -!- phoenix24(jfeb@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-20 15:05 zcommits: [zumastor commit] r1562 - trunk/benchmarks/nsnaps 2008-04-20 15:15 zcommits: zumastor b0.8.0 r1562 build success 2008-04-20 15:25 zcommits: zumastor b0.8.0 r1562 install success 2008-04-20 15:32 -!- MaZe(~MaZe@c-24-6-86-168.hsd1.ca.comcast.net) has joined #zumastor 2008-04-20 15:32 Bashisms in tests are bloody annoying. 2008-04-20 15:49 also 2008-04-20 15:50 RAW_DEV is undocumented in its usage within nsnaps 2008-04-20 15:53 well, undocumented in config.sh 2008-04-20 15:54 documented in README 2008-04-20 15:54 willn: what's the status of the hardy kernel? does it work fine? 2008-04-20 15:55 zcommits: [zumastor commit] r1563 - trunk/benchmarks/nsnaps 2008-04-20 16:03 pgquiles: Yeah. It runs, but hasn't been certified official 2008-04-20 16:05 willn: "not certified official" as in "it will not replace the current dapper kernel as soon as hardy is out but maybe later"? 2008-04-20 16:05 zcommits: zumastor b0.8.0 r1563 build success 2008-04-20 16:06 or "not certified official" as in "not tested with serious data"? 2008-04-20 16:12 as in, i'm really the only one using/testing the PPA packages 2008-04-20 16:16 zcommits: zumastor b0.8.0 r1563 install success 2008-04-20 16:56 zcommits: zumastor b0.8.0 r1562 test success 2008-04-20 17:14 http://compbrain.net/tmp/zumastor-tests.htm 2008-04-20 17:14 turns out my table layout logic is pretty silly 2008-04-20 18:17 -!- mech422(~steve@c-67-180-115-89.hsd1.ca.comcast.net) has joined #zumastor 2008-04-20 18:18 hi all 2008-04-20 18:18 I was interested in using Zumastor for providing replication/redundancy for iSCSI volumes (for use with Xen) 2008-04-20 18:19 Does anyone currently use it for this? Would zumastor be a good fit for this usage ? 2008-04-20 18:20 Howdy 2008-04-20 18:21 I'm not aware of anyone using it that way, and integration with Xen enabled kernels is somewhat limited. pgquiles has a kernel compiled with both Xen and zumastor patches. 2008-04-20 18:21 If you use it with iscsi exported volumes, you might be the first. 2008-04-20 18:27 willn: ahh - on the bright side, I won't be running zumastor on a xen kernel - I'll be accessing it via iscsi from xen... on the down side, being the first is always 'fun' :-) 2008-04-20 18:28 I actually found out about zumastor from the openfiler website - Does anyone here run openfiler ? 2008-04-20 18:31 zcommits: zumastor b0.8.0 r1563 test success 2008-04-20 18:42 I poked at it 2008-04-20 18:42 but not actively using it 2008-04-20 18:43 willn: May I ask why ? Was it just something you were playing with, or did it not meet some need you had ? 2008-04-20 18:45 I didn't have a specific use for it 2008-04-20 18:45 I just tend to try things out that catch my eye 2008-04-20 18:46 willn: ahh - I know the feeling :-) 2008-04-20 18:47 I'm looking at evaling openfiler for use at work. But so far, info. on the features I'd want to use seems scarce 2008-04-20 18:48 so then I started looking at zumastor to 'roll-my-own' 2008-04-20 18:50 what features were you looking at? 2008-04-20 18:51 willn: iscsi target, and replication/redundancy ... 2008-04-20 18:51 basically, I want a storage setup that lets me keep my Xen vm Images 'safe'.. 2008-04-20 18:52 so if a server dies, the images are pulled from a 'backup' server 2008-04-20 18:53 this way, I have a nicely fault tolerant Xen setup - if a 'server' (dom0) dies, just start the domU on a different server using the disk image from the storage backend 2008-04-20 18:53 if a storage backend dies, images are available from the 'backup' storage server 2008-04-20 18:54 So 2008-04-20 18:54 I'd like to support 40 iscsi connections with a sustained xfer rate of 1MB/sec for each connection (~40MB/sec total) and probably 1TB of storage 2008-04-20 18:54 That would definately be a first, as zumastor would be managing a volume directly exported as an iscsi target 2008-04-20 18:55 willn: yeap - that would be the goal... lvm => zumastor => iscsi => Xen 2008-04-20 18:55 ah, i think of it the other way (arrows in the other direction) =p 2008-04-20 18:55 (the snapshoting is nice for backing up the images, but not critical ) 2008-04-20 18:56 so you'd have more than one machine with the data on it exported via iscsi? 2008-04-20 18:56 right - 2 'storage backends' (openfiler, zumastor, drdb, whatever) with _local_ RAID1 in a replicated setup... 2008-04-20 18:57 failover could be done using heartbeat, etc 2008-04-20 18:57 ah, alrighty 2008-04-20 18:57 so, in theory that would work. 2008-04-20 18:57 so to RAID and replicated 1Tb, I need 4TB of space :-/ 2008-04-20 18:57 luckily sata disks are cheap. 2008-04-20 18:58 if your doing scsi, then ouch. 2008-04-20 18:58 willn: and _slow_ :-( but still faster then I'm figuring iscsi will be :-P 2008-04-20 18:58 where sata kills me is backup speeds... 2008-04-20 18:59 (which is another area replication could help with - I could pull the 'backup' storage server offline, and run backups against that... then put it back online and resync with the master... ) 2008-04-20 18:59 Get a whole bunch of fast small capacity drives, its almost like having one fast large capacity drive =] 2008-04-20 18:59 Yep. Thats how 'large customers' of mysql tend to do backups, replicate to a slave, offline the slave, make a lvm or other snapshot 2008-04-20 19:00 willn: yeah - but that only gets you so far with SATA - most machines only support 2 or 4 SATA drives, then you need to buy additional controllers - and 'small' sata drives aren't really much cheaper then 'large' one 2008-04-20 19:00 True. 2008-04-20 19:01 seems like when you get above 7.2k rpm, the largest capacity you will find is ~320G 2008-04-20 19:01 I was _really_ hoping not to have to do this myself - there's too many moving parts - replication (drdb, zumastor, etc), redundancy (heartbeat, vrouter), etc etc 2008-04-20 19:02 but openfiler seems to have stalled a bit - version 2.3 is still in beta, but was expected to ship last october 2008-04-20 19:02 You can buy commercial hardware, if you've got serious cash 2008-04-20 19:02 heh - not that serious 2008-04-20 19:02 but I don't even know how well their iscsi failover works 2008-04-20 19:02 just the drives alone would cost me like $1k if I roll my own 2008-04-20 19:03 with a commercial nas/san, that's gonna go up to like $6K minimum 2008-04-20 19:03 Yup 2008-04-20 19:03 gah.. 2008-04-20 19:03 http://code.google.com/p/ganeti/ -- There is that 2008-04-20 19:03 Doesn't give you awesome failover, but its passable 2008-04-20 19:04 yeah - seems like iscsi would be 'cleaner' then their drdb setup though... 2008-04-20 19:05 drdb means _every_ xen dom0 server needs enough disk space for _all_ possible xen domU images ? 2008-04-20 19:05 They have the advantage that its all rolled in to their management package 2008-04-20 19:06 They provision it such that each xen instance is replicated somewhere else (atleast once) 2008-04-20 19:06 (for instance - I have 40 vm images at 20G each - so I have ~ 800G of vmimages that ganeti would want to replicate to _all_ dom0 servers ? ) 2008-04-20 19:06 each image gets a master host and a replica host, iirc 2008-04-20 19:06 oh ? so its not as disk intensive as I thought - but not as robust either.. 2008-04-20 19:06 Right 2008-04-20 19:07 Its one of those cost/effort/feature balances 2008-04-20 19:07 Hmm ... I think I'd rather eat the disk space... 2008-04-20 19:07 yeah 2008-04-20 19:07 thats why I'm wanting to go iscsi and redundant storage backends... 2008-04-20 19:07 any server could run any vm - but I only need 2 copies of the data 2008-04-20 19:09 In theory ;). I know of some iscsi horror stories, but on the whole haven't seen it used enough to judge either way 2008-04-20 19:09 I can even live with losing the iscsi state when things fail - as long as the 'backup' storage server takes over within say 30 seconds, I can reboot the domU's 2008-04-20 19:09 yeah - especially with linux... I've heard its iscsi target support is less then stellar ? 2008-04-20 19:09 I can't speak to it. 2008-04-20 19:10 I actually thought about setting up some open solaris boxes with ZFS for the storage stuff - but I know nada about solaris 2008-04-20 19:10 in most cases I see people using HBAs and fancy SANs 2008-04-20 19:11 Heh - wish I had that sort of money :-) 2008-04-20 19:12 oh well - time to feed the kids :-) 2008-04-20 19:12 willn: thanks for the help :-) 2008-04-20 19:15 No worries. Let us know if you give it a shot 2008-04-20 19:55 -!- mech422(~steve@c-67-180-115-89.hsd1.ca.comcast.net) has left #zumastor 2008-04-20 20:02 kgdb merged... finally after all these years 2008-04-20 20:06 http://compbrain.net/misc/computing/nsnaps-test2-04-20-2008.png 2008-04-20 20:06 interesting patterns 2008-04-20 20:14 willn, the numbers are varying there a little more than they should 2008-04-20 20:14 for example, raw at snap 11 2008-04-20 20:17 I just started a 100 test run 2008-04-20 20:18 or 100 snaps 2008-04-20 20:18 in reality, I should do the 100 test run 10 times then combine plots irc.oftc.net #zumastor log beginning Mon Apr 21 00:00:01 PDT 2008 2008-04-21 00:00 willn, you forgot to rm the new manpages in make clean 2008-04-21 00:00 willn, a rare oversight ;-) 2008-04-21 00:10 oh dear, whatever shall the world do? 2008-04-21 00:10 heh 2008-04-21 00:10 Did you finish un-stubbing them? 2008-04-21 00:28 Are you prepping a patch to mail this week? 2008-04-21 00:32 zcommits: [zumastor commit] r1564 - trunk/ddsnap 2008-04-21 00:34 ACTION sleeps for work tommorow 2008-04-21 00:37 zcommits: zumastor b0.8.0 r1564 build success 2008-04-21 00:47 willn, (when you wake up) I have not unstubbed them and it is unlikely I will get time to work on man pages this week 2008-04-21 00:47 I am prepping patch for merging the optimization work 2008-04-21 00:47 and refactoring 2008-04-21 00:47 zcommits: zumastor b0.8.0 r1564 install success 2008-04-21 00:56 -!- erwan_taf(~erwan@81.80.43.67) has joined #zumastor 2008-04-21 02:17 zcommits: zumastor b0.8.0 r1564 test failure 5 2008-04-21 02:28 -!- pgquiles_(~pgquiles@62.43.226.52) has joined #zumastor 2008-04-21 03:09 -!- pgquiles__(~pgquiles@209.Red-88-18-198.staticIP.rima-tde.net) has joined #zumastor 2008-04-21 03:36 time to say goodnight... 2008-04-21 03:36 in a sense 2008-04-21 06:39 -!- pgquiles(~pgquiles@230.Red-79-153-249.staticIP.rima-tde.net) has joined #zumastor 2008-04-21 07:53 oh poop 2008-04-21 07:53 I don't think the reiser fail was real.. 2008-04-21 07:59 http://feeds.engadget.com/~r/weblogsinc/engadget/~3/274672118/ 2008-04-21 07:59 Perhaps we should aquire some of those 2008-04-21 08:54 zcommits: [zumastor commit] r1565 - trunk/benchmarks/nsnaps 2008-04-21 09:04 zcommits: zumastor b0.8.0 r1565 build success 2008-04-21 09:09 zcommits: zumastor b0.8.0 r1565 install success 2008-04-21 10:17 -!- MaZe(~MaZe@216-239-45-4.google.com) has joined #zumastor 2008-04-21 10:42 -!- jiayingz(~jiayingz@207.47.98.129.static.nextweb.net) has joined #zumastor 2008-04-21 10:45 zcommits: zumastor b0.8.0 r1565 test success 2008-04-21 10:48 -!- phoenix24(srcm@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-21 11:18 -!- dkegel(~chatzilla@207.47.98.129.static.nextweb.net) has joined #zumastor 2008-04-21 11:22 So, turns out we can only have 64 snapshots. 2008-04-21 11:22 heh 2008-04-21 11:23 The nsnaps test I was running failed for that reason 2008-04-21 11:23 why would it fail? 2008-04-21 11:23 could not create snapshot because unable to read message header (Broken pipe) 2008-04-21 11:23 unable to create snapshot id 65 2008-04-21 11:24 ah that is intentional? 2008-04-21 11:24 I wasn't planning on it 2008-04-21 11:39 -!- MaZe(~MaZe@216-239-45-4.google.com) has joined #zumastor 2008-04-21 11:45 zcommits: [zumastor commit] r1566 - trunk/cbtb/host-scripts 2008-04-21 11:55 zcommits: zumastor b0.8.0 r1566 build success 2008-04-21 11:56 -!- CIA-5(~CIA@208.69.182.149) has joined #zumastor 2008-04-21 12:00 zbot: rss-watch zbuild 300 2008-04-21 12:00 watcher thread started 2008-04-21 12:00 irc bot juggling, yay 2008-04-21 12:01 -!- phoenix24(jwgzoiy@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-21 12:20 zbuild: zumastor b0.8.0 r1566 install success 2008-04-21 13:03 -!- phoenix24(zgxq@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-21 13:26 Hi everyone! 2008-04-21 13:47 hello 2008-04-21 13:50 ACTION pokes cia 2008-04-21 13:50 zbuild: zumastor b0.8.0 r1566 test success 2008-04-21 13:51 -!- DavidSev(david@lolcat.mercenariesguild.net) has left #zumastor 2008-04-21 14:05 zbuild: zumastor b0.8.0 r1567 build success 2008-04-21 14:06 how can I verify shadow copies are exported OK? I'd say they are not working :-/ 2008-04-21 14:07 03williamanowak * r1568 10/trunk/cbtb/host-scripts/ (test-zuma-dapper-i386.sh zuma-dapper-i386.sh): * Fix bashisms [let => $(())] in cbtb host-scripts 2008-04-21 14:07 huzzah 2008-04-21 14:08 half the people in the channel are bots :-D 2008-04-21 14:08 pgquiles: They are more fun to talk to ;) 2008-04-21 14:09 the moment they start to actually commit code, I'll run scared :-D 2008-04-21 14:09 pgquiles: re: shadow copies 2008-04-21 14:10 your not seeing the previous versions show up in windows explorer? 2008-04-21 14:11 and the volume is exported in samba with 'vfs objects = shadow_copy' ? 2008-04-21 14:11 correct 2008-04-21 14:11 and no hidden directories with lots of @@ either 2008-04-21 14:12 I cannot remember the format of those directories but I do remember lots of @ symbols :-) 2008-04-21 14:12 odd thing is when I created the volumes I used the --samba option and the /var/lib/zumastor/.../samba file contains "yes" 2008-04-21 14:13 I'm using samba 3.0.28a (latest) 2008-04-21 14:13 hold 2008-04-21 14:15 zbuild: zumastor b0.8.0 r1568 build success 2008-04-21 14:16 zbuild: zumastor b0.8.0 r1567 install success 2008-04-21 14:16 Can you post the export volume bit from smb.conf 2008-04-21 14:17 sure, wait a minute while I VPN to work 2008-04-21 14:17 feel free to pastebin it 2008-04-21 14:21 -!- pgquiles_(~pgquiles@62.43.226.52.static.user.ono.com) has joined #zumastor 2008-04-21 14:21 http://rafb.net/p/h2BD3j41.html 2008-04-21 14:21 willn: it's a very simple samba share where everybody can read and write 2008-04-21 14:22 ACTION <3 0777 :) 2008-04-21 14:24 in /var/run/zumastor/mount/zumamultimedia, do you have @GMT-* directories? 2008-04-21 14:24 no 2008-04-21 14:24 what is zumastor --version? 2008-04-21 14:25 now it's 0.8.0 rev 1561 but I created it with rev 1543 2008-04-21 14:26 zbuild: zumastor b0.8.0 r1568 install success 2008-04-21 14:26 is mountsnap set to yes? (/var/lib/zumastor/volumes/zumamultimedia/filesystem/mountsnap) 2008-04-21 14:29 swell, 2 SoC slots for Zumastor 2008-04-21 14:34 willn: sorry, I had not seen your question. Yes, mountsnap = yes 2008-04-21 14:35 -!- phoenix24(xthy@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-21 14:36 hmm 2008-04-21 14:36 That is very strange 2008-04-21 14:36 willn: that's exactly the reason I'm asking ;-) 2008-04-21 14:36 Are any snapshots mounted? 2008-04-21 14:36 no 2008-04-21 14:36 only the latest one 2008-04-21 14:37 Can you take a snapshot for posterity 2008-04-21 14:38 working on that 2008-04-21 14:39 the only unusual thing with these volumes is they are not actually in use but rsync'ed from the actual volumes, which are in other server and are not zumastored 2008-04-21 14:39 but I'd rsync -av server::vol /var/run/zumastor/mount/zumamultimedia does not destroy data 2008-04-21 14:42 ahh 2008-04-21 14:43 is that bad, doctor? 2008-04-21 14:50 I missed the question? 2008-04-21 14:51 yeah, when I said I was rsync'ing to the zumastor volumes, you said "ahh". I thought that meant you've found the reason snapshots are failing for me :-) 2008-04-21 14:51 oh nope ;) 2008-04-21 14:51 Just acknowledging the situation 2008-04-21 14:51 :-( 2008-04-21 14:51 Do we know if any snapshots exist currently? 2008-04-21 14:52 still being taken, it's a 100G volume :-) 2008-04-21 14:52 on HW RAID-6 with SATA 7.2KRPM disks 2008-04-21 14:52 Well, in theory once that finishes you should have the snapshot mounted for shadow copy export 2008-04-21 14:53 I hope so but the fact that has not worked for the last 3-4 days makes me cautious 2008-04-21 14:53 It doesn't sound like there are any snapshots currently 2008-04-21 14:54 there should be, snapshots are taken daily and files are rsync'ed daily, too 2008-04-21 14:54 and I know for sure some files have changed 2008-04-21 14:54 replication is working perfect, btw 2008-04-21 14:55 replication is working but the snapshotting isnt? 2008-04-21 14:56 yes 2008-04-21 14:56 does master.log have anything to say on the subject 2008-04-21 14:57 nothing unusual: http://rafb.net/p/sgX2gS20.html 2008-04-21 14:58 that stop is due to a server reboot (just to test zumastor re-mounted everything fine after a reboot) 2008-04-21 14:59 It doesn't look like its finishing snapshot creation 2008-04-21 15:00 You said your doing daily snapshots? 2008-04-21 15:00 yes 2008-04-21 15:00 does /var/lib/zumastor/volumes/zumamultimedia/master/snapshots/daily 2008-04-21 15:00 it's not working either on smaller (25GB) volumes 2008-04-21 15:00 have a list of snapnumbers in it? 2008-04-21 15:01 that file does not exist, I guess you are right about unfinished snapshots 2008-04-21 15:01 :-/ 2008-04-21 15:01 did you do zumastor define schedule? 2008-04-21 15:02 yes 2008-04-21 15:03 root@spectrum:~# zumastor define schedule zumamultimedia --daily 30 2008-04-21 15:03 is that reflected in /var/lib/zumastor/volumes/zumamultimedia/master/schedule/daily 2008-04-21 15:03 it is 2008-04-21 15:03 well poop. 2008-04-21 15:07 Anything in server.log for the volume? 2008-04-21 15:08 or agent for that matter 2008-04-21 15:08 nothing in server.log 2008-04-21 15:08 nothing in agent.log 2008-04-21 15:09 nothing recent, I mean 2008-04-21 15:10 do you get "Mon Apr 21 18:10:17 2008: [9629] create_snapshot: Create snapshot tag = 2, bit = 1) 2008-04-21 15:10 in server.log for the supposed snapshot numbrs? 2008-04-21 15:12 no 2008-04-21 15:12 no create_snapshot entry 2008-04-21 15:12 last one is from April 20th 2008-04-21 15:13 mmm re-reading the zumastor howto, I see there is no "zumastor define master zumamultimedia --samba" in the target but I did execute that both on the origin and on the replica, might that be the problem? 2008-04-21 15:14 It shouldnt, --samba really just adds the bind mount with a special name, iirc 2008-04-21 15:16 so, is a "zumastor define master" needed both in the origin and in the replicas? or only in the origin? 2008-04-21 15:16 define master should only occur on the origin 2008-04-21 15:18 mmm according to my notebook, I only ran define master on the origin but I'd say I ran it on the replica, too 2008-04-21 15:22 hmm. 2008-04-21 15:22 I'm puzzled 2008-04-21 15:22 me too 2008-04-21 15:22 hi pgquiles_ 2008-04-21 15:23 could u post what 'zumastor status' shows? 2008-04-21 15:23 when I created the volumes, I took a few snapshots and I'd swear snapshots were mounted. I did not try shadow copies back then, though 2008-04-21 15:23 jiayingz: hi 2008-04-21 15:24 http://rafb.net/p/ImzQl157.html 2008-04-21 15:24 that's on the origin, for all the volumes 2008-04-21 15:24 pgquiles_, could you check if /etc/cron.daily/zumastor exists and its mode? 2008-04-21 15:24 jiayingz: it does exist but it's not executable! 2008-04-21 15:24 644 2008-04-21 15:25 i think it need to be executable 2008-04-21 15:25 same on the replica 2008-04-21 15:25 all the other files in /etc/cron.daily are executable, I'm setting 755 2008-04-21 15:26 i guess that will solve the problem 2008-04-21 15:26 That should be fixed with recent packaging 2008-04-21 15:26 jiayingz: thank you 2008-04-21 15:26 willn: thanks to you, too! 2008-04-21 15:26 np 2008-04-21 15:26 But that shouldn't affect why zumastor snapshot daily wasn't running 2008-04-21 15:27 willn: do you mean the one I executed about an hour ago? 2008-04-21 15:27 yea 2008-04-21 15:27 may it be that it logs its activity when it finishes? :-? 2008-04-21 15:35 03williamanowak * r1569 10/trunk/zumastor/bin/zumastor: 2008-04-21 15:35 Wrap the read call within read_and_inc in a subfunction, and redirect its call 2008-04-21 15:35 stderr to /dev/null. This prevents bogus errors from showing up in zumastor 2008-04-21 15:35 logs. (We catch failures and return 0 anyway) 2008-04-21 15:35 Are the ddsnap processes churning cpu? 2008-04-21 15:38 hmm. 2008-04-21 15:39 willn: no 2008-04-21 15:39 cpu's are 99% idle 2008-04-21 15:39 You said your using 1561, for ddsnap and zumastor? 2008-04-21 15:40 yes 2008-04-21 15:41 but I created the volumes with 1543 and updated to 1561 just today 2008-04-21 15:42 ACTION peeks the diff of 1543 and 1561 2008-04-21 15:42 oh 2008-04-21 15:42 hmm 2008-04-21 15:43 does /sbin/nblock_write exist? 2008-04-21 15:43 it's there 2008-04-21 15:44 does /lib/zumastor/nblock_write exist? 2008-04-21 15:44 it does not 2008-04-21 15:45 grep nblock /lib/zumastor/common 2008-04-21 15:45 is it calling /sbin or /lib/zumastor 2008-04-21 15:45 it calls /sbin/nblock_write 2008-04-21 15:45 well, there goes that theory ;) 2008-04-21 15:46 but you might be right, after all 2008-04-21 15:46 hmm? 2008-04-21 15:46 remember I updated to 1561 about 12 hours ago, no scheduled snapshots have been taken with 1561 2008-04-21 15:46 those files might have not been there in 1543 2008-04-21 15:46 Did zumastor stop/start between upgrades? 2008-04-21 15:47 yes, it's stopped and started fine 2008-04-21 15:47 in 1543 there was /lib/zumastor/nblock_write 2008-04-21 15:47 Right. 2008-04-21 15:48 I was thinking if it hadnt stopped/started proper, it might be running in memory with the old path 2008-04-21 15:48 oh, no, I don't think so 2008-04-21 15:49 in a minute or two, 1569 should finish building gutsy style on the ppa 2008-04-21 15:49 ok 2008-04-21 15:49 I'll try the upgrade and see what happens 2008-04-21 15:50 can I control-c zumastor snapshot? it didn't to to background :-? 2008-04-21 15:50 "didn't go to" 2008-04-21 15:50 if nothing is happening, i'd say yes? 2008-04-21 15:51 killed 2008-04-21 15:56 huh 2008-04-21 15:56 Ive reproduced the problem. 2008-04-21 15:56 what was it? 2008-04-21 16:06 zbuild: [PPA zumastor-team] Accepted: zumastor 0.8.0-r1569 (source) 2008-04-21 16:06 zbuild: zumastor b0.8.0 r1569 build success 2008-04-21 16:06 zbuild: [PPA zumastor-team] Accepted: ddsnap 0.8.0-r1569 (source) 2008-04-21 16:06 zbuild: zumastor b0.8.0 r1567 test success 2008-04-21 16:07 -!- pgquiles_(~pgquiles@230.Red-79-153-249.staticIP.rima-tde.net) has joined #zumastor 2008-04-21 16:10 pgquiles_: If you stop zumastor, then start it, snapshots should work again 2008-04-21 16:11 we just sampled the issue on test hardware 2008-04-21 16:11 zbuild: zumastor b0.8.0 r1569 install success 2008-04-21 16:11 "snapshots" meaning "zumastor snapshot ..." or "snapshots should be mounted again"? 2008-04-21 16:11 well, things probably wont work until you do that 2008-04-21 16:12 zumastor snapshot specifically, but I assume that hanging means other things dont work too 2008-04-21 16:12 ok 2008-04-21 16:12 I rebooted the machines, anyway, just to make sure the proper version was in memory 2008-04-21 16:12 a bit extreme but... :-) 2008-04-21 16:12 :) 2008-04-21 16:13 I wanted to place those machines in production by the end of the week but we have a meeting on Thursday and may decide to upgrade to Hardy, which would mean a few more days of testing 2008-04-21 16:15 pgquiles_: http://code.google.com/p/zumastor/issues/detail?id=119 2008-04-21 16:18 willn: thanks 2008-04-21 16:18 willn: what was the actual reason for zumastor to hang? 2008-04-21 16:19 not sure yet. investigating 2008-04-21 16:20 ok 2008-04-21 16:23 zissues: Issue 119 in zumastor: Upgrading zumastor packages leaves zumastor non-functional 2008-04-21 16:27 Did snapshots kick in? 2008-04-21 16:27 -!- dkegel(~chatzilla@207.47.98.129.static.nextweb.net) has joined #zumastor 2008-04-21 16:27 wb dkegel 2008-04-21 16:27 willn: I've not tried 2008-04-21 16:27 I'll VPN to work in a min 2008-04-21 16:31 jiayingz: One issue is that we install the init script twice. 2008-04-21 16:31 Once the debian way, once the scripts/install (barf) way 2008-04-21 16:31 willn, really 2008-04-21 16:32 make install? 2008-04-21 16:32 Oh, right. Was looking at ddsnap 2008-04-21 16:32 er 2008-04-21 16:32 no 2008-04-21 16:32 make install calls scripts/install 2008-04-21 16:32 Yuck. 2008-04-21 16:34 that is for 'make install' to work 2008-04-21 16:34 Yea 2008-04-21 16:34 -!- phoenix24(jioiixj@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-21 16:34 Its just not pretty =] 2008-04-21 16:34 true 2008-04-21 16:40 willn, didn't the upgrade-issue #119 gets solved with proposed workaround ? 2008-04-21 16:41 I wonder why does web-interface is readme most of the time :(( 2008-04-21 17:02 they're chasing a bug 2008-04-21 17:17 -!- pgquiles__(~pgquiles@62.43.226.52.static.user.ono.com) has joined #zumastor 2008-04-21 17:18 willn: workaround works fine, snapshots work again 2008-04-21 17:20 -!- pgquiles__(~pgquiles@230.Red-79-153-249.staticIP.rima-tde.net) has joined #zumastor 2008-04-21 17:21 zbuild: zumastor b0.8.0 r1568 test success 2008-04-21 17:23 phoenix24: No, the problem still exist, that if you upgrade things break 2008-04-21 17:23 Granted, its easy to un-break, but I don't like breaking things. 2008-04-21 17:41 zbuild: zumastor b0.8.0 r1570 build failure 1 2008-04-21 17:41 zbuild: [PPA zumastor-team] Accepted: zumastor 0.8.0-r1570 (source) 2008-04-21 17:42 zbuild: [PPA zumastor-team] Accepted: ddsnap 0.8.0-r1570 (source) 2008-04-21 17:42 -!- phoenix24(mxpx@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-21 17:59 -!- MaZe(~MaZe@216-239-45-4.google.com) has left #zumastor 2008-04-21 18:07 03williamanowak * r1570 10/trunk/zumastor/ (4 files in 2 dirs): (log message trimmed) 2008-04-21 18:07 This is an attempt to resolve issues present with our packaging. If this model 2008-04-21 18:07 works, i'll apply the same recipie to ddsnap. Packages built should look the 2008-04-21 18:07 same. 2008-04-21 18:07 - Do not use scripts/install. It is ugly. 2008-04-21 18:07 - Let dh_installman deal with manpages, no need to gzip ourselves 2008-04-21 18:07 - Let dh_installcron deal with cronjobs, no need to install ourselves 2008-04-21 18:07 03williamanowak * r1571 10/trunk/builddepends.sh: Add rsync to builddepends.sh since the builder cannot currently read package dependencies in the control file. 2008-04-21 18:14 well, that didnt help 119 2008-04-21 18:36 pgquiles: Out of curiosity, did you use an apt repository to install 1561? 2008-04-21 18:38 I'm going to wager a guess yess 2008-04-21 18:38 since using dpkg to install the package by hand, everything works after that. 2008-04-21 18:39 Somehow apt is causing the init script to not work properly 2008-04-21 18:42 zbuild: zumastor b0.8.0 r1571 install success 2008-04-21 18:42 zbuild: zumastor b0.8.0 r1571 build success 2008-04-21 18:42 zbuild: [PPA zumastor-team] Accepted: zumastor 0.8.0-r1571 (source) 2008-04-21 18:42 zbuild: [PPA zumastor-team] Accepted: ddsnap 0.8.0-r1571 (source) 2008-04-21 18:59 -!- phoenix24(vpbx@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-21 19:02 jiayingz, ping 2008-04-21 19:02 zbuild: zumastor b0.8.0 r1569 test success 2008-04-21 19:05 http://blog.makezine.com/breakfast_with_reflow.jpg 2008-04-21 19:34 hi flips 2008-04-21 19:36 jayingz, query chat... => 2008-04-21 19:59 -!- phoenix24_(mdjhfn@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-21 20:37 zbuild: zumastor b0.8.0 r1571 test success 2008-04-21 21:00 -!- phoenix24(fzlvnm@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-21 21:40 Why are different versions built for Hardy(r1538) & Gusty(r1571) at the ppa.launchpad.net for zumastor/ddsnap/kernel ? 2008-04-21 21:43 -!- MaZe(~MaZe@c-24-6-86-168.hsd1.ca.comcast.net) has joined #zumastor 2008-04-21 22:00 -!- phoenix24(gknp@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-21 23:30 phoenix24: Launchpad does not let you send the same version for building on different releases 2008-04-21 23:30 flips: shapor: Can we get ops to change the topic from time to time? 2008-04-21 23:30 yes, and we can set it so everybody can change it 2008-04-21 23:31 -!- phoenix24(hlijv@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-21 23:33 -!- ChanServ changed mode/#zumastor -> +o willn 2008-04-21 23:38 03Daniel.Raymond.Phillips * r1572 10/trunk/ddsnap/ (Makefile ddsnapd.c): (log message trimmed) 2008-04-21 23:38 miscellaneous cleanups: 2008-04-21 23:38 * Freshens the "to do" at the top of ddsnapd.c (which should be moved 2008-04-21 23:38 to a proper todo file or maybe eliminated completely) 2008-04-21 23:38 * Removed hexdump 2008-04-21 23:38 * unit testing code is removed from ddsnapd.c. It will be re-added in a 2008-04-21 23:38 better form in a later patch in this series. irc.oftc.net #zumastor log beginning Tue Apr 22 00:00:01 PDT 2008 2008-04-22 00:07 phoenix24: Launchpad does not let you send the same version for building on different releases 2008-04-22 00:07 zbuild: zumastor b0.8.0 r1572 install success 2008-04-22 00:08 zbuild: zumastor b0.8.0 r1572 build success 2008-04-22 00:08 zbuild: [PPA zumastor-team] Accepted: zumastor 0.8.0-r1572 (source) 2008-04-22 00:08 zbuild: [PPA zumastor-team] Accepted: ddsnap 0.8.0-r1572 (source) 2008-04-22 01:21 willn: yes, I did use the zumastor-team apt repository 2008-04-22 01:39 -!- phoenix24(rorhzqh@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-22 01:48 zbuild: zumastor b0.8.0 r1572 test success 2008-04-22 02:39 -!- phoenix24_(kzmgbn@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-22 03:03 the ddsnap/patches are for kernel versio 2.6.24.2 or older; what how do I apply patches to 2.6.24-13 or later.. as is done in the ppa build ? 2008-04-22 03:05 the 2.6.24.2 patches should apply fine 2008-04-22 03:07 while applying patch to linus's tree I get the following error : http://paste.ubuntu.com/7731/ 2008-04-22 03:38 03Daniel.Raymond.Phillips * r1573 10/trunk/ddsnap/Makefile: remove extra binary from make clean 2008-04-22 03:40 -!- phoenix24(pzmxlv@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-22 03:42 how, whose bot is CIA-5? 2008-04-22 03:42 Dont know. 2008-04-22 03:42 anyway, the cool think about my latest commit is, it was commit straight from a git repository 2008-04-22 03:43 no subversion repository involved 2008-04-22 03:43 err 2008-04-22 03:43 no subversion checkout involved 2008-04-22 03:43 (subversion always has a central repository, which is the point) 2008-04-22 03:44 So, I better start using git-repo for experimentation.. rather svn. 2008-04-22 03:46 this is how I cloned the subversion repo: git-svn clone --username daniel.raymond.phillips https://zumastor.googlecode.com:443/svn 2008-04-22 03:47 I don't think the :443 is actually needed 2008-04-22 03:47 that gets the whole thing, all the branches etc 2008-04-22 03:47 nice! 2008-04-22 03:47 there's some fairly ugly history in there, like when somebody checked in an 11 MB svg file 2008-04-22 03:47 and some debs were checked in 2008-04-22 03:48 those must be over 2 weeks old history though. 2008-04-22 03:48 several months old, but there is no way to tell svn to forget it 2008-04-22 03:49 everybody who wants to use git-svn to push/pull has to grab all that old cruft onto their machine 2008-04-22 03:49 fortunately, git compresses it nicely 2008-04-22 03:50 38 MB for the repo, though it really should be half that size 2008-04-22 03:51 oh, that includes all the web stuff 2008-04-22 03:51 including various binary files like jpgs etc 2008-04-22 03:51 so it's not too bad 2008-04-22 03:52 damn, stayed up too late again 2008-04-22 03:55 as i checked, in linux-kernel-2.6.25/block/ll_rw_blk.c is probably merged. 2008-04-22 03:56 and I guess, It'll need a patch. So let me see if I can do it :) 2008-04-22 03:57 good luck 2008-04-22 04:08 zbuild: zumastor b0.8.0 r1573 build success 2008-04-22 04:08 zbuild: [PPA zumastor-team] Accepted: zumastor 0.8.0-r1573 (source) 2008-04-22 04:08 zbuild: [PPA zumastor-team] Accepted: ddsnap 0.8.0-r1573 (source) 2008-04-22 04:17 well zbuild is really putting on a good show 2008-04-22 04:18 zbuild: zumastor b0.8.0 r1573 install success 2008-04-22 04:19 flips: working around the clock? :-) 2008-04-22 04:20 you too? 2008-04-22 04:20 it's almost lunch time here 2008-04-22 04:20 I guess I will do some of the old sleeping until noon 2008-04-22 04:20 night 2008-04-22 04:50 -!- phoenix24(rstqz@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-22 05:44 zbuild: zumastor b0.8.0 r1573 test success 2008-04-22 08:40 pgquiles: So that bug is apt specific 2008-04-22 08:40 no matter how you build the .debs, if you use apt, that bug occurs 2008-04-22 08:41 I'm going to try some init script mashing to fix things today 2008-04-22 08:48 willn: ok 2008-04-22 08:49 willn: I had never hit a bug happening only when apt is used 2008-04-22 08:50 It's a first for me as well. 2008-04-22 08:51 But I am pretty sure its related to the environment in which the init script is called 2008-04-22 08:51 mostly because our init scripts are insane 2008-04-22 08:57 -!- phoenix24(bqlssoo@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-22 10:00 -!- phoenix24(chlqbx@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-22 10:10 zissues: Issue 120 in zumastor: race in snapshot creation 2008-04-22 10:41 03williamanowak * r1574 10/trunk/cbtb/tests/1/zero-zumastor.sh: 2008-04-22 10:41 - Expect this to fail 2008-04-22 10:41 - Implement test side of new test interface 2008-04-22 10:41 - This will start the flow of CBTB test refactoring to the tests can be 2008-04-22 10:41 parameterized to run on any test harness. 2008-04-22 10:41 - Do a tiny bit of cleanup 2008-04-22 10:59 zbuild: zumastor b0.8.0 r1574 build success 2008-04-22 10:59 zbuild: [PPA zumastor-team] Accepted: zumastor 0.8.0-r1574 (source) 2008-04-22 10:59 zbuild: [PPA zumastor-team] Accepted: ddsnap 0.8.0-r1574 (source) 2008-04-22 11:07 03williamanowak * r1575 10/trunk/ (buildcurrentsrc.sh zumastor/debian/control buildcurrent.sh): Stop using sed to update _VER_ in the control file as it modifies the working directory 2008-04-22 11:09 zbuild: zumastor b0.8.0 r1574 install success 2008-04-22 11:29 zbuild: [PPA zumastor-team] Accepted: zumastor 0.8.0-r1575 (source) 2008-04-22 11:30 zbuild: [PPA zumastor-team] Accepted: ddsnap 0.8.0-r1575 (source) 2008-04-22 11:40 zbuild: zumastor b0.8.0 r1575 build success 2008-04-22 11:50 zbuild: zumastor b0.8.0 r1575 install success 2008-04-22 12:50 zbuild: zumastor b0.8.0 r1574 test success 2008-04-22 13:04 -!- tim_vimm(~Tim@cpe-76-90-98-247.socal.res.rr.com) has joined #zumastor 2008-04-22 13:13 ok, random cleanups are in, now to merge a non-trivial patch 2008-04-22 14:25 -!- tim_vimm(~Tim@cpe-76-90-98-247.socal.res.rr.com) has joined #zumastor 2008-04-22 14:30 zbuild: zumastor b0.8.0 r1575 test success 2008-04-22 14:39 -!- tim_vimm(~Tim@cpe-76-90-98-247.socal.res.rr.com) has joined #zumastor 2008-04-22 14:51 03williamanowak * r1576 10/trunk/cbtb/host-scripts/ (test-zuma-dapper-i386.sh runtests.sh): 2008-04-22 14:51 - Update the CBTB harness to support the new CBTB/test interface 2008-04-22 14:51 - runtests.sh gets new ENV Variables 2008-04-22 14:51 - test-zuma-dapper-i386 gets logic to read DEV[0-9]SIZE from the test 2008-04-22 15:00 zbuild: [PPA zumastor-team] Accepted: zumastor 0.8.0-r1576 (source) 2008-04-22 15:00 zbuild: [PPA zumastor-team] Accepted: ddsnap 0.8.0-r1576 (source) 2008-04-22 15:10 zbuild: zumastor b0.8.0 r1576 build success 2008-04-22 15:20 zissues: Issue 121 in zumastor: Replication may stop when snapshots are squashed on downstream 2008-04-22 15:21 zbuild: zumastor b0.8.0 r1576 install success 2008-04-22 15:25 hi flips 2008-04-22 16:11 so. 2008-04-22 16:11 im flustered 2008-04-22 16:20 -!- cbsmith(~user@207.47.98.129.static.nextweb.net) has joined #zumastor 2008-04-22 16:56 zbuild: zumastor b0.8.0 r1576 test success 2008-04-22 18:05 pgquiles: Were you using gutsy when you had the issue yesterday? 2008-04-22 21:23 willn, because? 2008-04-22 21:28 jiayingz and I have only been able to get the issue to occur on gutsy or higher machines upgrading zumastor with apt-get or one of its friends (aptitude) 2008-04-22 21:28 This is why bash daemons are evil. 2008-04-22 21:29 (hah, evil daemons...) 2008-04-22 23:56 -!- phoenix24(llv@h1119083.serverkompetenz.net) has joined #zumastor irc.oftc.net #zumastor log beginning Wed Apr 23 00:00:01 PDT 2008 2008-04-23 00:36 Hi everyone! 2008-04-23 00:36 I guess most people are asleep. 2008-04-23 00:54 Yea 2008-04-23 00:54 ABout that time 2008-04-23 00:54 :) 2008-04-23 00:55 -!- phoenix24(ypo@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-23 00:58 I dont think issue 119 is getting fixed tonight. 2008-04-23 00:58 Good night. 2008-04-23 00:58 willn, I couldn't replay the #119 for myself. 2008-04-23 01:14 willn: yes, I was using Gutsy 2008-04-23 01:15 willn: but I'm using an updated version of bash with the latest patches (it's in my PPA) 2008-04-23 01:55 -!- phoenix24_(lmbuf@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-23 02:56 -!- phoenix24(ajkw@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-23 03:56 -!- phoenix24(rugifjl@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-23 06:02 -!- phoenix24(dvcqss@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-23 07:15 -!- phoenix24(xjdmwm@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-23 09:13 pgquiles: Ok, We're suspecting a change in apt from dapper to gutsy, as we can't reproduce the issue on dapper. 2008-04-23 09:18 willn: ok 2008-04-23 09:18 willn: does issue #119 happen in hardy? 2008-04-23 09:19 visual c++ 2008 is not able to open visual c++ 4.2 projects... so much for backwards compatibility :-/ 2008-04-23 09:35 pgquiles: Thats on my list to try today. We've written a script to reproduce on gutsy.. 2008-04-23 10:25 -!- MaZe(~MaZe@216-239-45-4.google.com) has joined #zumastor 2008-04-23 10:32 -!- tim_vimm(~Tim@cpe-76-90-98-247.socal.res.rr.com) has joined #zumastor 2008-04-23 10:48 -!- cbsmith(~user@207.47.98.129.static.nextweb.net) has joined #zumastor 2008-04-23 10:54 Okay, so I'm apparently missing something rather obvious here re: deadlock w/delta generation. 2008-04-23 10:54 Why would the client deadlock? It should be outside the path for any page fault handling, no? 2008-04-23 12:04 -!- phoenix24(qdbh@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-23 12:12 zissues: Issue 31 in zumastor: btree code has no unit test 2008-04-23 12:18 ACTION is back (gone for 00:13.04) 2008-04-23 12:22 Okay, I now grok the client deadlock issue. 2008-04-23 12:42 zissues: Issue 122 in zumastor: NFS latency measurement utility 2008-04-23 12:52 -!- MaZe(~MaZe@216-239-45-4.google.com) has left #zumastor 2008-04-23 13:14 -!- tim_vimm(~Tim@cpe-76-90-98-247.socal.res.rr.com) has joined #zumastor 2008-04-23 13:19 -!- phoenix24(bek@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-23 13:34 -!- tim_vimm(~Tim@cpe-76-90-98-247.socal.res.rr.com) has joined #zumastor 2008-04-23 13:49 quiet today 2008-04-23 13:49 -!- willn changed topic to "http://www.zumastor.org | http://code.google.com/p/zumastor | Next release number 0.8.0" 2008-04-23 13:55 ACTION is back (gone for 00:51.01) 2008-04-23 14:59 -!- phoenix24(eeoifw@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-23 15:23 03williamanowak * r1577 10/trunk/cbtb/host-scripts/ (test-zuma-dapper-i386.sh continuous-build.sh): 2008-04-23 15:23 - patch for cbtb/host-scripts/test-zuma-dapper-i386.sh to fix 2008-04-23 15:23 the non-passing of device names. 2008-04-23 15:23 - gzip logs that the builder creates in 2008-04-23 15:23 cbtb/host-scripts/continuous-build.sh 2008-04-23 15:37 zbuild: [PPA zumastor-team] Accepted: zumastor 0.8.0-r1577 (source) 2008-04-23 15:37 zbuild: [PPA zumastor-team] Accepted: ddsnap 0.8.0-r1577 (source) 2008-04-23 15:58 zbuild: zumastor b0.8.0 r1577 install success 2008-04-23 16:04 -!- phoenix24(asemyv@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-23 16:19 -!- flipz(~phlipz@216.239.45.19) has joined #zumastor 2008-04-23 16:33 -!- dkegel(~chatzilla@207.47.98.129.static.nextweb.net) has joined #zumastor 2008-04-23 16:35 jiaying, are you still waiting for patch review? 2008-04-23 16:38 jiayingz, ping 2008-04-23 16:39 dkegel, I know about it ;-) 2008-04-23 16:39 yes, she's still waiting, we did my two first 2008-04-23 16:39 hi flipz 2008-04-23 16:39 dkegel, hi 2008-04-23 16:40 amazing that guy using ddsnap for root volume 2008-04-23 16:40 check out my latest commit, it came from git, where git is acting as a subversion client 2008-04-23 16:40 really 2008-04-23 16:40 I don't fully follow 2008-04-23 16:40 I will try to catch up with him 2008-04-23 16:40 and he found a bug 2008-04-23 16:40 crazy people are useful :-) 2008-04-23 16:40 I never for the life of me knew what that special status command was for 2008-04-23 16:40 _really_ bad design to use status that way 2008-04-23 16:41 we need to keep him happy so he finds more bugs for us 2008-04-23 16:41 oh yes 2008-04-23 16:41 I hope he took it the right way when I suggested he make the patch 2008-04-23 16:41 hi dkegel 2008-04-23 16:41 hi j 2008-04-23 16:41 after what he's been through, it should be easy by comparison 2008-04-23 16:42 yes, i am waiting for reviews for issue 46, issue 113, issue 114, and issue 121 2008-04-23 16:42 the patches are all attached to the issues? 2008-04-23 16:43 yes 2008-04-23 16:43 flipz, what's going on with your mail client? The last few messages you've sent to the list are strange, doing reply all fails 2008-04-23 16:44 It's like you bcc'd the list?! 2008-04-23 16:49 dkegel, it's not my mail client 2008-04-23 16:49 somebody added a bogus cc 2008-04-23 16:49 03Daniel.Raymond.Phillips * r1578 10/trunk/ddsnap/ (ddsnap.c ddsnapd.c ddsnap.h): 2008-04-23 16:49 Refactor snapshot store initialization to be more readable and to expose 2008-04-23 16:49 init_super and init_journal methods for allocation and journal replay unit 2008-04-23 16:49 testing, respectively. 2008-04-23 16:49 ? 2008-04-23 16:49 the headers in the mail as sent from here do not have that cc 2008-04-23 16:49 it got added on the other side 2008-04-23 16:49 there is _no_ rewriting going on in my mta 2008-04-23 16:51 Can you send me the headers of a message you sent to the list recently? 2008-04-23 16:51 yes 2008-04-23 16:51 If it's not your mail client, how come this problem only affects messages you sent to the list? 2008-04-23 16:52 oh wait 2008-04-23 16:52 I see it 2008-04-23 16:52 it was my client 2008-04-23 16:52 whew 2008-04-23 16:52 it only happened once 2008-04-23 16:53 then kept getting cc'd 2008-04-23 16:54 funny, jiaying's fwd came through to me with a cc: to undisclosed recipeints 2008-04-23 16:54 that has to be a goof from her side 2008-04-23 16:55 That's what happens when we reply all to your original bogus message, maybe it infected her 'f' command :-) 2008-04-23 16:57 however it happened, it ended up being about two hours of engineering for a trivial one line change :-/ 2008-04-23 16:57 oh well 2008-04-23 16:58 the patch as a whole was done more efficiently than that 2008-04-23 16:58 flips, what do you think about the prospects for integrating ddsnap into lvm2? (First with ddsnapd still in userspace, but perhaps later moving it into kernel module, otherwise people will be surprised by the userspace daemon...?) 2008-04-23 16:58 dkegel, good prospects 2008-04-23 16:59 we have to prove the stability beyond any doubt, which we have pretty much done in our own minds 2008-04-23 16:59 drake was looking around for an existing project that ddsnap could fit into and generate lots of users, but he overlooked the obvious candidate... 2008-04-23 16:59 but that has to be generally agreed 2008-04-23 16:59 I think the team here agrees, now we just have to do it and demonstrate the stability and performance... 2008-04-23 16:59 they say people talk about ddsnap on lvm-devel 2008-04-23 16:59 damn, one more list to monitor 2008-04-23 17:00 the traditional way is to have unrelated users asking on the lvm list for this 2008-04-23 17:00 it's happening 2008-04-23 17:00 indeed 2008-04-23 17:00 you can lead a horse to water, but you have to wait until it's thirsty until it will drink ;-) 2008-04-23 17:00 they've been asking for the feature for several years... 2008-04-23 17:00 now they're asking for our code. 2008-04-23 17:00 about a year now 2008-04-23 17:00 ben was the first to ask 2008-04-23 17:01 because he knows what it can do vs the old cllient 2008-04-23 17:01 one's a fluke, two's a trend 2008-04-23 17:01 for me, the watershed point where I am willing to go directly offer it is, when the metaupdate runs at the same speed as old lvm 2008-04-23 17:01 old lvm snapshot 2008-04-23 17:02 right now, it is just a little slower in theory, though people report it is faster in practice 2008-04-23 17:02 looking forward to will finishing the performance cluster harness 2008-04-23 17:02 yes 2008-04-23 17:02 now, that's going to be quite a few steps 2008-04-23 17:02 first, finishing merging this one 2008-04-23 17:02 there are two more cleanup patches to go in before the actual optimization 2008-04-23 17:03 then... the optimization needs to be re-engineered slightly in a awy that works for the index update 2008-04-23 17:03 probably we want to demonstrate more than 1% speedup before committing actual optimization... 2008-04-23 17:03 I will write a mail asbout that 2008-04-23 17:03 there are a couple of _very_ subtle issues in doing that 2008-04-23 17:03 I think the 1% indicates there is a mistake in the patch 2008-04-23 17:04 yup 2008-04-23 17:04 but I have not had time yet to track it down, been fully occupied with merging the support patches 2008-04-23 17:04 which has taken a frightening amount of time, hopefully that will accelerate 2008-04-23 17:05 having git working as a subversion client is a big deal, it gives me a way of putting patches waiting for ack aside, that I never had before 2008-04-23 17:05 I used to just put them in a directory, then not be able to know for sure which is current 2008-04-23 17:05 very fragile 2008-04-23 17:05 now each patch in progress gets a git-svn branch 2008-04-23 17:05 I can diff against it efficiently 2008-04-23 17:06 I usually only have one or two live at any time. Meatball programmers live in a different world. 2008-04-23 17:06 you'd have to diff against diffs to get a similar effect with raw patches.... 2008-04-23 17:06 I am juggling 5-6 in this one area alone, at the moment 2008-04-23 17:06 hopefully it will be down to 3 in an hour or so 2008-04-23 17:06 or less 2008-04-23 17:07 -!- tim_vimm(~Tim@cpe-76-90-98-247.socal.res.rr.com) has joined #zumastor 2008-04-23 17:07 I wonder if the undisclosed receipients was oroginaly caused by me sending a mail to zumastor@google.com this morning 2008-04-23 17:07 I suspect so 2008-04-23 17:07 So, plan is: cleanup, one unit test, 0.8, optimization, benchmarks, lvm2 integration, profit? 2008-04-23 17:08 the lvm2 integration is not our decision, but we can take steps to make it more likely 2008-04-23 17:08 zbuild: [PPA zumastor-team] Accepted: ddsnap 0.8.0-r1578 (source) 2008-04-23 17:08 well, we can do trial patches that show what it would look like. Then make nice with the lvm2 guys... 2008-04-23 17:09 otherwise, that is the plan... except after bechmarks goes... reroll bitmap optimization to introduce the concept of "deferred dirty buffers"; write the index optimization under similar lines; test; benchmark; merge; merge; merge 2008-04-23 17:10 invite ben to come post on our list some more would be a good move 2008-04-23 17:10 maybe invite him to refresh his socket patch to use the new kernel api 2008-04-23 17:10 oh, and "fix any zumastor bugs found by crazy lvm2 users" 2008-04-23 17:10 or explain to us he alreadyu did, and we lost it... 2008-04-23 17:11 more users = more bugs found no doubt 2008-04-23 17:11 we've been pretty lucky having only superficial ones from pgquiles 2008-04-23 17:11 we seem to be ahead of him in deploying 2008-04-23 17:11 ? 2008-04-23 17:13 he said he was doing it right then 2008-04-23 17:13 probably has 2008-04-23 17:13 I just didn't ask him 2008-04-23 17:14 but we've been live for some time... 2008-04-23 17:14 ... hence that bug Jiaying found the other day when downstream was squashing... 2008-04-23 17:15 "him" = pgquiles? 2008-04-23 17:15 yes 2008-04-23 17:15 we're certtainly ahead, as we should be 2008-04-23 17:15 I take nothing for granted... 2008-04-23 17:15 ... except perhaps my wife's amazing ability to cook ... 2008-04-23 17:16 ... and the free chocolate here... 2008-04-23 17:17 Hmm, should we have an office pool on when we'll be ready to present merge proposal to lvm2 guys 2008-04-23 17:17 maybe drake could provide the prize from that brewery he drives by... 2008-04-23 17:28 any takers? 2008-04-23 17:33 zbuild: zumastor b0.8.0 r1577 test success 2008-04-23 17:33 lessee... 0.8 May 2nd, optimizations & benchmarks in 0.9 on June 13th, 2nd pass on optimization & benchmarks in 0.10 on July 25th, merge merge merge in 0.11 on Sept 5th. 2008-04-23 17:33 So I bet Sept 5th.: 2008-04-23 17:35 $ svn update 2008-04-23 17:35 svn: PROPFIND request failed on '/svn/trunk' 2008-04-23 17:35 svn: PROPFIND of '/svn/trunk': 502 Bad Gateway (https://zumastor.googlecode.com) 2008-04-23 17:35 I could go beat up Jason but he's already feeling beat up 2008-04-23 17:36 oh, what the hell, I'll beat him up anyway 2008-04-23 17:37 done 2008-04-23 17:42 well I think I committed the diskio change then immediately uncommited it 2008-04-23 17:42 03Daniel.Raymond.Phillips * r1579 10/trunk/ddsnap/ (diskio.h ddsnap.c ddsnapd.c diskio.c): Move diskio helpers to diskio.c 2008-04-23 17:42 too many sparks flying from your fingertips? 2008-04-23 17:42 from somewhere ;-) 2008-04-23 17:45 how does one read those new fangled diffs on code.google.com 2008-04-23 17:45 I see red regions of code, but no explanation of what a red region is 2008-04-23 17:45 I would much prefer traditional diff 2008-04-23 17:46 bah gui diff 2008-04-23 17:46 flipz: It's fairly simple... 2008-04-23 17:46 red = delete 2008-04-23 17:46 green = add 2008-04-23 17:46 its done by a standard lib, same as trac and friends 2008-04-23 17:47 hmm so it looks like I really did back out the diskio commit 2008-04-23 17:48 zbuild: [PPA zumastor-team] Accepted: zumastor 0.8.0-r1578 (source) 2008-04-23 17:48 zbuild: zumastor b0.8.0 r1578 install success 2008-04-23 17:53 zbuild: [PPA zumastor-team] Accepted: ddsnap 0.8.0-r1579 (source) 2008-04-23 17:55 oh wait, my mistake, it looks like the commit went just fine 2008-04-23 17:55 r1579 | Daniel.Raymond.Phillips | 2008-04-23 17:40:32 -0700 (Wed, 23 Apr 2008) | 2 lines 2008-04-23 17:55 Move diskio helpers to diskio.c 2008-04-23 17:56 I was just looking in the wrong file 2008-04-23 17:56 fd_size really was supposed to be deleted from ddsnap.c 2008-04-23 17:58 zbuild: [PPA zumastor-team] Accepted: zumastor 0.8.0-r1579 (source) 2008-04-23 17:59 well that is enough merge madness for now 2008-04-23 17:59 go have lunch :-) 2008-04-23 17:59 skip breakfast? 2008-04-23 18:00 03williamanowak * r1580 10/trunk/cbtb/host-scripts/continuous-build.sh: If bfiles is zero length, don't use commas. 2008-04-23 18:12 flipz, what does 'host zumastor.googlecode.com' say for you now? 2008-04-23 18:12 d'oh 2008-04-23 18:23 zbuild: [PPA zumastor-team] Accepted: zumastor 0.8.0-r1580 (source) 2008-04-23 18:23 zbuild: [PPA zumastor-team] Accepted: ddsnap 0.8.0-r1580 (source) 2008-04-23 18:28 zbuild: zumastor b0.8.0 r1580 build failure 255 2008-04-23 20:40 -!- flips(~phlipz@adsl-63-202-13-187.dsl.snfc21.pacbell.net) has joined #zumastor 2008-04-23 20:45 -!- MaZe(~MaZe@216-239-45-4.google.com) has joined #zumastor 2008-04-23 20:59 zbuild: zumastor b0.8.0 r1578 test failure 142 2008-04-23 21:02 blah 2008-04-23 21:56 -!- phoenix24(owbhju@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-23 23:38 -!- phoenix24(rshzkxk@h1119083.serverkompetenz.net) has joined #zumastor irc.oftc.net #zumastor log beginning Thu Apr 24 00:00:01 PDT 2008 2008-04-24 00:38 -!- phoenix24_(oomjkl@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-24 00:41 So 2008-04-24 00:41 I've sort of got mvsata working 2008-04-24 01:16 the change of init_work is causing much grief 2008-04-24 01:25 pMail sent 2008-04-24 01:25 I should go home and sleep now. 2008-04-24 02:01 -!- erwan_taf(~erwan@81.80.43.67) has joined #zumastor 2008-04-24 03:10 -!- pgquiles(~pgquiles@62.43.226.52.static.user.ono.com) has joined #zumastor 2008-04-24 03:20 -!- phoenix24(skukqu@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-24 04:21 -!- phoenix24(fmedrs@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-24 04:41 -!- MaZe(~MaZe@c-24-6-86-168.hsd1.ca.comcast.net) has joined #zumastor 2008-04-24 07:57 Hardy is releases 2008-04-24 08:31 -!- flips(~phlipz@adsl-63-202-13-187.dsl.snfc21.pacbell.net) has joined #zumastor 2008-04-24 09:19 willn: ping 2008-04-24 09:21 pong 2008-04-24 09:22 willn: is the zumastor-enabled hardy kernel in the PPA? I cannot find it (there is only an old version and only for LPIA) 2008-04-24 09:24 There should be 2008-04-24 09:24 oh hmm 2008-04-24 09:24 I think I was waiting for openvz fixes 2008-04-24 09:24 but they haven't gone upstream yet 2008-04-24 09:24 I'll do a build within the hour as upstream stands. 2008-04-24 09:25 ok 2008-04-24 09:26 did you decide to go production with hardy? 2008-04-24 09:29 -!- phoenix24(zlr@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-24 09:31 -!- flips(~phlipz@adsl-63-202-13-187.dsl.snfc21.pacbell.net) has joined #zumastor 2008-04-24 10:14 -!- flips(~phlipz@adsl-63-202-13-187.dsl.snfc21.pacbell.net) has joined #zumastor 2008-04-24 10:16 -!- cbsmith(~xman@pool-96-239-42-181.nycmny.east.verizon.net) has joined #zumastor 2008-04-24 10:17 -!- jiayingz(~jiayingz@207.47.98.129.static.nextweb.net) has joined #zumastor 2008-04-24 10:19 shapor. ping? 2008-04-24 10:30 willn: yes, production with hardy 2008-04-24 10:30 flips pong? 2008-04-24 10:35 groovy. 2008-04-24 10:52 here goes nothing 2008-04-24 10:52 flavormaker++ 2008-04-24 10:57 -!- MaZe(~MaZe@216-239-45-4.google.com) has joined #zumastor 2008-04-24 10:58 -!- phoenix24(vdjg@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-24 11:09 zbuild: [PPA zumastor-team] Accepted: linux 2.6.24-16.30~ppa4 (source) 2008-04-24 12:21 -!- flips(~phlipz@216.239.45.19) has joined #zumastor 2008-04-24 12:23 -!- mitchg(~mitchg@207.47.98.129.static.nextweb.net) has joined #zumastor 2008-04-24 12:37 -!- tim_vimm(~Tim@rrcs-64-183-50-58.west.biz.rr.com) has joined #zumastor 2008-04-24 12:39 -!- phoenix24(eruorhg@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-24 12:40 -!- cbsmith(~user@72-255-49-38.client.stsn.net) has joined #zumastor 2008-04-24 13:22 hmm 2008-04-24 13:22 ? 2008-04-24 13:23 Working on uml tests 2008-04-24 13:23 which dont work on amd64 2008-04-24 13:24 OH.... 2008-04-24 13:24 UML has "issues" with amd64 in the most recent kernels, IIRC 2008-04-24 13:25 The issue is that their are not any scripts in zumastor for testing on amd64 2008-04-24 13:25 willn: Actually, the "issues" are for running an x86 kernel on amd64 I think. 2008-04-24 13:29 03williamanowak * r1581 10/trunk/cbtb/tests/1/ (13 files): 2008-04-24 13:29 - Firstly, this change will break UML tests 2008-04-24 13:29 - Update all single node tests to use the new test/runner API 2008-04-24 13:29 - Tiny cleanups 2008-04-24 13:30 willn: Looks like 2.6.24 actually fixed the x86 on amd64 issue too, so I'm just totally out to lunch. 2008-04-24 13:46 i'm just going to keep patching things until stuff works 2008-04-24 13:50 zbuild: [PPA zumastor-team] Accepted: zumastor 0.8.0-r1581 (source) 2008-04-24 13:50 zbuild: [PPA zumastor-team] Accepted: ddsnap 0.8.0-r1581 (source) 2008-04-24 13:52 -!- natacha29(~natacha29@d033.dhcp212-198-248.noos.fr) has joined #zumastor 2008-04-24 13:54 Well 2008-04-24 13:54 I think buildcurrent is now arch independent 2008-04-24 13:55 zbuild: zumastor b0.8.0 r1581 build failure 255 2008-04-24 14:04 ACTION waits for the ppa kernels... 2008-04-24 14:05 03williamanowak * r1582 10/trunk/buildcurrent.sh: - Some fixes for building on amd64, does not affect i386 builds 2008-04-24 14:20 zbuild: [PPA zumastor-team] Accepted: zumastor 0.8.0-r1582 (source) 2008-04-24 14:20 zbuild: [PPA zumastor-team] Accepted: ddsnap 0.8.0-r1582 (source) 2008-04-24 14:22 -!- tim_vimm(~Tim@rrcs-64-183-50-58.west.biz.rr.com) has joined #zumastor 2008-04-24 14:22 ACTION is back (gone for 00:22.15) 2008-04-24 14:22 ACTION is back 2008-04-24 14:25 zbuild: zumastor b0.8.0 r1582 build success 2008-04-24 14:38 -!- phoenix24(xvhna@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-24 14:39 willn, while du you have a uml kernel patched with zumastor kernel patches .. while doing your test ? 2008-04-24 14:40 zbuild: zumastor b0.8.0 r1582 install success 2008-04-24 14:40 let me rephrase. 2008-04-24 14:40 do you use a uml-kernel patched with zumastor kernel patches for the testing ? 2008-04-24 14:47 -!- flips(~phlipz@216.239.45.19) has joined #zumastor 2008-04-24 14:48 phoenix24: Yes. 2008-04-24 14:48 Currretnly that only works on i386 2008-04-24 14:51 And are the tests run using different kernel versions/ diff. zumastor releases or for both combinations ? 2008-04-24 14:57 No, we are doing all testing on 2.6.24 currently 2008-04-24 14:57 and the latest zumastor/ddsnap revision 2008-04-24 15:38 -!- flips(~phlipz@216.239.45.19) has joined #zumastor 2008-04-24 16:14 -!- phoenix24(bianwzy@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-24 17:05 zbuild: zumastor b0.8.0 r1582 test failure 142 2008-04-24 17:08 -!- tim_vimm(~Tim@rrcs-64-183-50-58.west.biz.rr.com) has joined #zumastor 2008-04-24 17:16 03williamanowak * r1583 10/trunk/ (6 files in 3 dirs): 2008-04-24 17:16 - Make zumastor and ddsnap packages install manpages with dh_installman 2008-04-24 17:16 - Don't copy/gzip when someone else can do it for us 2008-04-24 17:16 - Stub out nblock_write manpage 2008-04-24 17:16 - Remove devspam manpage 2008-04-24 17:16 - Don't include devspam in the debian package 2008-04-24 17:28 03williamanowak * r1584 10/trunk/ddsnap/Makefile: Forgot to remove devspam.8 from makefile 2008-04-24 17:28 feh. 2008-04-24 17:40 zbuild: zumastor b0.8.0 r1583 build failure 1 2008-04-24 17:40 zbuild: [PPA zumastor-team] Accepted: zumastor 0.8.0-r1583 (source) 2008-04-24 17:41 zbuild: [PPA zumastor-team] Accepted: ddsnap 0.8.0-r1583 (source) 2008-04-24 17:44 ok, builds work again 2008-04-24 17:46 zbuild: [PPA zumastor-team] Accepted: zumastor 0.8.0-r1584 (source) 2008-04-24 17:46 ACTION is back (gone for 02:35.31) 2008-04-24 17:50 -!- cbsmith(~user@72-255-49-38.client.stsn.net) has left #zumastor 2008-04-24 17:51 zbuild: zumastor b0.8.0 r1584 build success 2008-04-24 18:11 zbuild: zumastor b0.8.0 r1584 install success 2008-04-24 18:16 -!- tim_vimm(~Tim@cpe-76-90-98-247.socal.res.rr.com) has joined #zumastor 2008-04-24 18:38 crap 2008-04-24 18:38 replication is broken 2008-04-24 18:46 zbuild: [PPA zumastor-team] Accepted: ddsnap 0.8.0-r1584 (source) 2008-04-24 19:02 flips: you need to double check your work 2008-04-24 19:59 zissues: Issue 123 in zumastor: replication stops because of a bug in fdsize 2008-04-24 20:25 03jiahotcake * r1585 10/trunk/ddsnap/ddsnap.c: 2008-04-24 20:25 Fix a bug introduced in revision 1579, see issue 123 2008-04-24 20:25 "replication stops because of a bug in fdsize". 2008-04-24 20:31 zbuild: zumastor b0.8.0 r1584 test failure 142 2008-04-24 20:41 zbuild: [PPA zumastor-team] Accepted: ddsnap 0.8.0-r1585 (source) 2008-04-24 20:41 zbuild: [PPA zumastor-team] Accepted: zumastor 0.8.0-r1585 (source) 2008-04-24 20:47 03jiahotcake * r1586 10/trunk/benchmarks/nsnaps/nsnaps.sh: Fix the ddsnap cleanup in nsnaps benchmark. 2008-04-24 20:56 03drake.diedrich * r1587 10/trunk/cbtb/tests/2/replication-zumastor.sh: Port replication test to new device names and pragmas. 2008-04-24 21:01 zbuild: [PPA zumastor-team] Accepted: zumastor 0.8.0-r1586 (source) 2008-04-24 21:01 zbuild: [PPA zumastor-team] Accepted: ddsnap 0.8.0-r1586 (source) 2008-04-24 21:02 zbuild: zumastor b0.8.0 r1585 build success 2008-04-24 21:12 zbuild: zumastor b0.8.0 r1586 build success 2008-04-24 21:12 zbuild: [PPA zumastor-team] Accepted: ddsnap 0.8.0-r1587 (source) 2008-04-24 21:12 zbuild: [PPA zumastor-team] Accepted: zumastor 0.8.0-r1587 (source) 2008-04-24 21:12 zbuild: zumastor b0.8.0 r1585 install success 2008-04-24 21:22 zbuild: zumastor b0.8.0 r1586 install success 2008-04-24 21:27 zbuild: zumastor b0.8.0 r1587 build success 2008-04-24 21:32 zbuild: zumastor b0.8.0 r1587 install success 2008-04-24 21:33 -!- flips(~phlipz@216.239.45.19) has joined #zumastor 2008-04-24 21:34 sha;por, there? 2008-04-24 22:20 flips: Did you see r1585 2008-04-24 22:27 willn, no 2008-04-24 22:27 I'll check 2008-04-24 22:28 If bfiles is zero length, don't use commas.? 2008-04-24 22:29 er 2008-04-24 22:30 no 2008-04-24 22:30 Fix a bug introduced in revision 1579, see issue 123 2008-04-24 22:30 "replication stops because of a bug in fdsize". 2008-04-24 22:30 You broke replication with r1579 2008-04-24 22:30 whoops 2008-04-24 22:34 I can't read those fscking colorized diffs 2008-04-24 22:34 what is the actual diff? 2008-04-24 22:38 How can you not? 2008-04-24 22:39 lines 1166 and 1170, != -> == 2008-04-24 22:40 I see a bright red on ! 2008-04-24 22:40 I don't see any = replacing the ! 2008-04-24 22:40 Its bright red 2008-04-24 22:40 because it was removed 2008-04-24 22:40 look at the next column 2008-04-24 22:40 and where is the = that replaces it? 2008-04-24 22:40 it works alot better when your code doesnt have stupidly long lines 2008-04-24 22:40 use the scrollbar, and move it to the right 2008-04-24 22:40 oh I see 2008-04-24 22:41 yes right 2008-04-24 22:41 or, you can use svn diff -r 1584:1585 if you prefer old school 2008-04-24 22:41 trying to convince git to show me the relevant diff 2008-04-24 22:41 well that is a project for tomorrow 2008-04-24 22:42 did you autotest catch it or what? 2008-04-24 22:42 our autotest I mean 2008-04-24 22:42 I was trying to setup replcation, and it failed everywhere 2008-04-24 22:42 also, the cbtb tests failed 2008-04-24 22:42 good 2008-04-24 22:42 that is what they are for 2008-04-24 22:42 so if you had run the uml tests you would have seen it before you blew up the trunk 2008-04-24 22:42 escaped the attention of all our eyes 2008-04-24 22:42 zbuild: zumastor b0.8.0 r1585 test success 2008-04-24 22:42 give me a uml test that works and I will run it 2008-04-24 22:42 show me a broken one and ill show you how to fix it 2008-04-24 22:43 command line to run one? 2008-04-24 22:43 trunk/cbtb/uml/setup.sh once 2008-04-24 22:43 willn, the reason we have autotesting is so we can "blow things up" 2008-04-24 22:43 trunk/cbtb/uml/smoke-etch.sh 2008-04-24 22:44 If you want to be responsible you should test before you commit 2008-04-24 22:44 we didn't put 2 man years in that just to try and do it's work for it 2008-04-24 22:44 ok, I'll try it 2008-04-24 22:44 it failed badly last time 2008-04-24 22:45 so you have to let me know when you think it is ready 2008-04-24 22:45 until then 2008-04-24 22:45 work toether 2008-04-24 22:45 As it stands, its i386 only 2008-04-24 22:45 do not criticize for people making mistakes 2008-04-24 22:45 people are human 2008-04-24 22:45 Please run it, if it does not work, let drake or I know asap and we will get it fixed 2008-04-24 22:45 and we can't spend all our time gazing at code hoping to cahtch the last mistake 2008-04-24 22:45 that is what computers are for 2008-04-24 22:45 I am not a computer 2008-04-24 22:45 are you? 2008-04-24 22:46 I didnt look at code to find the bug, I tried a simple setup and it failed. 2008-04-24 22:46 good 2008-04-24 22:46 that is the purpose of having tests 2008-04-24 22:46 no, this wasnt a test, this was me trying to get some production stuff running 2008-04-24 22:46 ah 2008-04-24 22:46 jiayingz looked at it and found the bug thanfully 2008-04-24 22:46 well we should install production from tagged releases 2008-04-24 22:46 don't you think? 2008-04-24 22:46 trunk is not for production 2008-04-24 22:46 trunk is devel 2008-04-24 22:47 I wasnt releasing my setup to production, but certifying a machine with the impending 0.8 release 2008-04-24 22:48 Admitedly, I have an advantage that all my changes are simple bash things usually. I have issues when parts of ddsnap break and I am up poop creek when it comes to troubleshooting them 2008-04-24 22:48 src/git/zumastor/trunk/cbtb/uml$ smoke-etch.sh 2008-04-24 22:48 bash: smoke-etch.sh: command not found 2008-04-24 22:48 ./smoke-etch ? 2008-04-24 22:49 ok 2008-04-24 22:49 it's trying 2008-04-24 22:49 You ran setup first, yes/ 2008-04-24 22:49 got to install dnsmasq 2008-04-24 22:49 no 2008-04-24 22:50 You need to run setup first. 2008-04-24 22:50 its a run once command, then the rest of your smoke-etch runs wont require that first 2008-04-24 22:51 $ sh setup.sh 2008-04-24 22:51 sudo: /tmp/tmp.UMOXjr4331: command not found 2008-04-24 22:53 Can you copy your output and mail me 2008-04-24 22:53 Will you be in tommorow? 2008-04-24 22:55 I will be in mtv tomorrow 2008-04-24 22:55 but I will do that 2008-04-24 22:55 can you install from branchs/0.7 next time please? 2008-04-24 22:55 trunk is for test and dev 2008-04-24 22:56 do we have a way of packaging from a branch? 2008-04-24 22:56 We only have what we post 2008-04-24 22:56 which is? 2008-04-24 22:56 My 'production' testing is really development, so hey 2008-04-24 22:56 sure 2008-04-24 22:56 that is good 2008-04-24 22:56 I will do everything I can to make the uml work for you 2008-04-24 22:56 when we package though, do we package from a branch? 2008-04-24 22:57 thanks 2008-04-24 22:57 If you promise to try and use it :) 2008-04-24 22:57 and I will do everything I can to use it 2008-04-24 22:57 oh yes 2008-04-24 22:57 great. 2008-04-24 22:57 see my last try 2008-04-24 22:57 spent 4 hours late a night 2008-04-24 22:57 I'll get on that tommorow asap. 2008-04-24 22:57 hit a hard roadblock 2008-04-24 22:57 Thakns for reporting it not working 2008-04-24 22:57 np 2008-04-24 22:57 good night 2008-04-24 22:57 I foolishly thought I could get mv_sata working, that was ~4 hours of work lost 2008-04-24 22:57 closed donw the googleplex here tonight 2008-04-24 22:58 right 2008-04-24 22:58 nice try 2008-04-24 22:58 but it's not quite that simple 2008-04-24 22:58 I will sit down and track the upstream changes on monday 2008-04-24 22:58 soon enough for you? 2008-04-24 22:59 No reasonable timeframe on mv_sata, its lowest priority 2008-04-24 22:59 I was going based on what I could google on the conversion 2008-04-24 23:00 docs on the INIT_WORK converstion are sadly lacking 2008-04-24 23:00 should take me about an hour to figure out when I'm back home 2008-04-24 23:00 with a real screen etc 2008-04-24 23:00 that is true 2008-04-24 23:00 in fact most internal kernel interfaces are poorly documented 2008-04-24 23:01 that is called "full employment contract for kernel hackers" 2008-04-24 23:01 unfortunately I just get pissed off by it 2008-04-24 23:01 I'd rather be employed to think 2008-04-24 23:01 than to know quasi secret interfaces 2008-04-24 23:05 I also am under educated in the ways of writing C 2008-04-24 23:05 so its all shooting in the dark for me 2008-04-24 23:05 anyways, i should sleep so I can get to work tommorow 2008-04-24 23:05 Goodnight all 2008-04-24 23:05 it was a nice try 2008-04-24 23:05 I'll show you how to track it through 2008-04-24 23:05 when I get back 2008-04-24 23:31 -!- pgquiles(~pgquiles@62.43.226.52.static.user.ono.com) has joined #zumastor 2008-04-24 23:39 willn: you've hit the "abi missing" error again in the hardy kernel :-) irc.oftc.net #zumastor log beginning Fri Apr 25 00:00:01 PDT 2008 2008-04-25 00:22 zbuild: zumastor b0.8.0 r1587 test failure 2 2008-04-25 00:41 -!- MaZe(~MaZe@216-239-45-4.google.com) has joined #zumastor 2008-04-25 00:47 -!- flips(~phlipz@adsl-63-202-13-187.dsl.snfc21.pacbell.net) has joined #zumastor 2008-04-25 05:40 03cbsmith * r1588 10/trunk/ddsnap/ddsnapd.c: 2008-04-25 05:40 Moved get_status to its own message. 2008-04-25 05:40 Status message size is now checked consistently with other message handling code. 2008-04-25 05:58 zbuild: [PPA zumastor-team] Accepted: zumastor 0.8.0-r1588 (source) 2008-04-25 05:58 zbuild: [PPA zumastor-team] Accepted: ddsnap 0.8.0-r1588 (source) 2008-04-25 06:08 zbuild: zumastor b0.8.0 r1588 build success 2008-04-25 06:18 zbuild: zumastor b0.8.0 r1588 install success 2008-04-25 07:40 -!- tim_vimm(~Tim@cpe-76-90-98-247.socal.res.rr.com) has joined #zumastor 2008-04-25 07:48 zbuild: zumastor b0.8.0 r1588 test failure 2 2008-04-25 09:09 -!- tim_vimm(~Tim@cpe-76-90-98-247.socal.res.rr.com) has joined #zumastor 2008-04-25 09:22 -!- tim_vimm(~Tim@cpe-76-90-98-247.socal.res.rr.com) has joined #zumastor 2008-04-25 10:03 Ack 2008-04-25 10:16 03drake.diedrich * r1589 10/trunk/cbtb/tests/2/replication-zumastor.sh: 2008-04-25 10:16 This is failing due to device name propagation changes. 2008-04-25 10:16 Expect it to fail and add extra debugging. 2008-04-25 10:32 -!- dkegel(~chatzilla@207.47.98.129.static.nextweb.net) has joined #zumastor 2008-04-25 10:43 03jiahotcake * r1590 10/trunk/zumastor/bin/zumastor: 2008-04-25 10:43 Trap HUP signal in 'zumastor master' daemon so that it does 2008-04-25 10:43 not exit during apt-get updgrade, see issue 119. 2008-04-25 10:44 zbuild: zumastor b0.8.0 r1589 install success 2008-04-25 10:44 zbuild: zumastor b0.8.0 r1589 build success 2008-04-25 10:44 zbuild: [PPA zumastor-team] Accepted: ddsnap 0.8.0-r1589 (source) 2008-04-25 10:44 zbuild: [PPA zumastor-team] Accepted: zumastor 0.8.0-r1589 (source) 2008-04-25 11:04 zbuild: zumastor b0.8.0 r1590 build success 2008-04-25 11:04 zbuild: [PPA zumastor-team] Accepted: zumastor 0.8.0-r1590 (source) 2008-04-25 11:04 zbuild: [PPA zumastor-team] Accepted: ddsnap 0.8.0-r1590 (source) 2008-04-25 11:14 zbuild: zumastor b0.8.0 r1590 install success 2008-04-25 11:20 -!- flips(~phlipz@adsl-63-202-13-187.dsl.snfc21.pacbell.net) has joined #zumastor 2008-04-25 11:22 -!- tim_vimm(~Tim@cpe-76-90-98-247.socal.res.rr.com) has joined #zumastor 2008-04-25 12:02 03jiahotcake * r1591 10/trunk/benchmarks/nsnaps/nsnaps.sh: Change nsnaps script to let 'ddsnap server' write its output to a logfile. 2008-04-25 12:08 -!- MaZe(~MaZe@216-239-45-4.google.com) has joined #zumastor 2008-04-25 12:24 zbuild: zumastor b0.8.0 r1589 test success 2008-04-25 12:29 zbuild: zumastor b0.8.0 r1591 install success 2008-04-25 12:29 zbuild: zumastor b0.8.0 r1591 build success 2008-04-25 12:30 zbuild: [PPA zumastor-team] Accepted: zumastor 0.8.0-r1591 (source) 2008-04-25 12:30 zbuild: [PPA zumastor-team] Accepted: ddsnap 0.8.0-r1591 (source) 2008-04-25 12:58 03williamanowak * r1592 10/trunk/cbtb/uml/setup.sh: Make sure the stagefile is executable 2008-04-25 13:10 zbuild: [PPA zumastor-team] Accepted: zumastor 0.8.0-r1592 (source) 2008-04-25 13:10 zbuild: [PPA zumastor-team] Accepted: ddsnap 0.8.0-r1592 (source) 2008-04-25 13:20 zbuild: zumastor b0.8.0 r1592 install success 2008-04-25 13:20 zbuild: zumastor b0.8.0 r1592 build success 2008-04-25 13:50 zbuild: zumastor b0.8.0 r1590 test success 2008-04-25 13:52 -!- flips(~phlipz@216.239.45.19) has joined #zumastor 2008-04-25 13:55 03williamanowak * r1593 10/trunk/cbtb/uml/ (debootstrap-etch-i386.sh smoke-etch.sh test-zuma-uml.sh): 2008-04-25 13:55 UML test harness fixup 2008-04-25 13:55 - Fix some bash issues 2008-04-25 13:55 - add DEV${num}NAME params 2008-04-25 14:05 03drake.diedrich * r1594 10/trunk/cbtb/tests/1/boot-order.sh: 2008-04-25 14:05 Add env to boot-order.sh script, to help in debugging test harnesses 2008-04-25 14:05 and seeing what variables are actually available to tests. 2008-04-25 14:05 (And repush the cbtb qemu test harness to zumabuild, in case something 2008-04-25 14:05 was missed in earlier rollouts) 2008-04-25 14:05 zbuild: [PPA zumastor-team] Accepted: zumastor 0.8.0-r1593 (source) 2008-04-25 14:05 zbuild: [PPA zumastor-team] Accepted: ddsnap 0.8.0-r1593 (source) 2008-04-25 14:10 zbuild: zumastor b0.8.0 r1593 build success 2008-04-25 14:11 03williamanowak * r1595 10/trunk/cbtb/uml/ (setup.sh build-etch-i386.sh): 2008-04-25 14:11 - Zumastor is a _all .deb now 2008-04-25 14:11 - Install a whole pile of dependencies first, so we can build tunbr and whatnot 2008-04-25 14:16 zbuild: [PPA zumastor-team] Accepted: zumastor 0.8.0-r1594 (source) 2008-04-25 14:16 zbuild: [PPA zumastor-team] Accepted: ddsnap 0.8.0-r1594 (source) 2008-04-25 14:26 zbuild: zumastor b0.8.0 r1594 build success 2008-04-25 14:29 There are so many things to chase down with this uml stuff 2008-04-25 14:29 03Daniel.Raymond.Phillips * r1596 10/trunk/ddsnap/ (ddsnap.c Makefile): Revmove ddraid from ddsnap makefile; change SVNVERSION to VERSION internal to ddsnap 2008-04-25 14:29 ugh. 2008-04-25 14:31 Now ddsnap --version output looks silly 2008-04-25 14:33 There was a reason it was SVNVERSION and not version, since that refered to the svn revision, not the ddsnap version. 2008-04-25 14:34 Post a patch reverting that part of it, adding comments explaining what's going on, and see what Dan says 2008-04-25 14:36 zbuild: zumastor b0.8.0 r1595 build success 2008-04-25 14:36 zbuild: [PPA zumastor-team] Accepted: zumastor 0.8.0-r1595 (source) 2008-04-25 14:36 zbuild: [PPA zumastor-team] Accepted: ddsnap 0.8.0-r1595 (source) 2008-04-25 14:36 zbuild: zumastor b0.8.0 r1593 install success 2008-04-25 14:42 dkegel: We had a hallway conference about it, he didn't agree. 2008-04-25 14:42 03williamanowak * r1597 10/trunk/cbtb/uml/ (setup.sh build-etch-i386.sh): 2008-04-25 14:42 - Fix more deps 2008-04-25 14:42 - Fix another missing _all replacement for zumastor 2008-04-25 14:43 Can you route around the damage? 2008-04-25 14:44 Ideally i'll just submit a patch that actually turns VERSION into a version and SVNVERSION back into what it was 2008-04-25 14:46 zbuild: [PPA zumastor-team] Accepted: ddsnap 0.8.0-r1596 (source) 2008-04-25 14:46 zbuild: zumastor b0.8.0 r1594 install success 2008-04-25 14:51 zbuild: zumastor b0.8.0 r1596 build success 2008-04-25 14:51 zbuild: zumastor b0.8.0 r1595 install success 2008-04-25 14:51 zbuild: [PPA zumastor-team] Accepted: zumastor 0.8.0-r1596 (source) 2008-04-25 14:57 zbuild: [PPA zumastor-team] Accepted: ddsnap 0.8.0-r1597 (source) 2008-04-25 15:07 zbuild: zumastor b0.8.0 r1597 build success 2008-04-25 15:07 zbuild: zumastor b0.8.0 r1596 install success 2008-04-25 15:07 zbuild: [PPA zumastor-team] Accepted: zumastor 0.8.0-r1597 (source) 2008-04-25 15:17 zbuild: zumastor b0.8.0 r1597 install success 2008-04-25 15:32 zbuild: zumastor b0.8.0 r1592 test success 2008-04-25 16:37 zbuild: [PPA zumastor-team] Accepted: linux 2.6.24-16.30~ppa5 (source) 2008-04-25 17:07 zbuild: zumastor b0.8.0 r1597 test success 2008-04-25 17:17 03williamanowak * r1598 10/trunk/cbtb/uml/setup.sh: Add /dev/net/tun permissions check to setup.sh 2008-04-25 17:27 zbuild: [PPA zumastor-team] Accepted: zumastor 0.8.0-r1598 (source) 2008-04-25 17:27 zbuild: [PPA zumastor-team] Accepted: ddsnap 0.8.0-r1598 (source) 2008-04-25 17:38 zbuild: zumastor b0.8.0 r1598 build success 2008-04-25 17:53 zbuild: zumastor b0.8.0 r1598 install success 2008-04-25 18:11 Sent out the fixit patch for review. 2008-04-25 19:28 -!- dank(~chatzilla@cpe-76-90-71-54.socal.res.rr.com) has joined #zumastor 2008-04-25 19:53 zbuild: zumastor b0.8.0 r1598 test success 2008-04-25 22:44 -!- jiayingz_(~jiayingz@cpe-76-167-221-105.socal.res.rr.com) has joined #zumastor 2008-04-25 22:54 -!- flips(~phillips@phunq.net) has joined #zumastor irc.oftc.net #zumastor log beginning Sat Apr 26 00:00:01 PDT 2008 2008-04-26 03:04 -!- MaZe(~MaZe@c-24-6-86-168.hsd1.ca.comcast.net) has joined #zumastor 2008-04-26 04:26 -!- pgquiles(~pgquiles@vpn022.vpns.upv.es) has joined #zumastor 2008-04-26 06:45 -!- pgquiles(~pgquiles@vpn089.vpns.upv.es) has joined #zumastor 2008-04-26 09:50 -!- pgquiles(~pgquiles@vpn085.vpns.upv.es) has joined #zumastor 2008-04-26 11:29 -!- pgquiles(~pgquiles@vpn085.vpns.upv.es) has joined #zumastor 2008-04-26 11:36 -!- jiayingz_(~jiayingz@cpe-76-167-221-105.socal.res.rr.com) has joined #zumastor 2008-04-26 13:37 -!- jiayingz_(~jiayingz@adsl-99-172-70-130.dsl.irvnca.sbcglobal.net) has joined #zumastor 2008-04-26 15:27 -!- jiayingz__(~jiayingz@adsl-99-172-70-130.dsl.irvnca.sbcglobal.net) has joined #zumastor 2008-04-26 17:48 -!- jiayingz_(~jiayingz@adsl-99-172-70-129.dsl.irvnca.sbcglobal.net) has joined #zumastor 2008-04-26 21:04 03williamanowak * r1599 10/trunk/doc/zumastor-howto.txt: Update documentation for the usable zumastor-team ppa 2008-04-26 21:19 zbuild: zumastor b0.8.0 r1599 build success 2008-04-26 21:19 zbuild: [PPA zumastor-team] Accepted: ddsnap 0.8.0-r1599 (source) 2008-04-26 21:19 zbuild: [PPA zumastor-team] Accepted: zumastor 0.8.0-r1599 (source) 2008-04-26 21:25 -!- jiayingz_(~jiayingz@adsl-99-172-70-129.dsl.irvnca.sbcglobal.net) has joined #zumastor 2008-04-26 21:34 zbuild: zumastor b0.8.0 r1599 install success 2008-04-26 21:55 -!- jiayingz__(~jiayingz@adsl-99-172-70-130.dsl.irvnca.sbcglobal.net) has joined #zumastor 2008-04-26 23:00 zbuild: Pipes Output 2008-04-26 23:05 zbuild: zumastor b0.8.0 r1599 test success irc.oftc.net #zumastor log beginning Sun Apr 27 00:00:01 PDT 2008 2008-04-27 01:53 -!- pgquiles(~pgquiles@vpn059.vpns.upv.es) has joined #zumastor 2008-04-27 04:04 -!- pgquiles(~pgquiles@vpn059.vpns.upv.es) has joined #zumastor 2008-04-27 11:26 -!- MaZe(~MaZe@c-24-6-86-168.hsd1.ca.comcast.net) has joined #zumastor 2008-04-27 12:05 -!- jiayingz_(~jiayingz@cpe-76-167-221-105.socal.res.rr.com) has joined #zumastor 2008-04-27 13:58 03williamanowak * r1600 10/trunk/cbtb/host-scripts/ (continuous-install.sh runtests.sh): Fix issues with multinode tests, this will require a refresh of the cbtb machine 2008-04-27 14:16 zbuild: [PPA zumastor-team] Accepted: zumastor 0.8.0-r1600 (source) 2008-04-27 14:16 zbuild: [PPA zumastor-team] Accepted: ddsnap 0.8.0-r1600 (source) 2008-04-27 14:21 zbuild: zumastor b0.8.0 r1600 build success 2008-04-27 14:25 -!- jiayingz_(~jiayingz@76.167.221.105) has joined #zumastor 2008-04-27 14:31 zbuild: zumastor b0.8.0 r1600 install success 2008-04-27 16:06 zbuild: zumastor b0.8.0 r1600 test success 2008-04-27 16:47 -!- jiayingz_(~jiayingz@76.167.221.105) has joined #zumastor 2008-04-27 19:08 -!- jiayingz_(~jiayingz@76.167.221.105) has joined #zumastor 2008-04-27 19:41 -!- jiayingz_(~jiayingz@76.167.221.105) has joined #zumastor irc.oftc.net #zumastor log beginning Mon Apr 28 00:00:01 PDT 2008 2008-04-28 01:35 -!- pgquiles(~pgquiles@105.Red-88-0-158.dynamicIP.rima-tde.net) has joined #zumastor 2008-04-28 07:37 -!- jiayingz_(~jiayingz@cpe-76-167-221-105.socal.res.rr.com) has joined #zumastor 2008-04-28 09:13 hi flips 2008-04-28 09:27 03drake.diedrich * r1601 10/trunk/cbtb/tests/2/replication-zumastor.sh: 2008-04-28 09:27 With the new runtests.sh, new-style variables in dual-node tests are 2008-04-28 09:27 expected to fail. Kick off a new build/test without EXPECT_FAIL in this test. 2008-04-28 09:32 hi jiaying! 2008-04-28 09:33 hi dkegel 2008-04-28 09:35 hello world 2008-04-28 09:36 We feeling a release a-comin'? 2008-04-28 09:37 still waiting for the review of my patches 2008-04-28 09:38 Are they attached to an issue? 2008-04-28 09:39 willn, yes 2008-04-28 09:40 #121 and? 2008-04-28 09:41 #113, #114, and #46 2008-04-28 09:43 I reviewed the change to zumastor, I don't think I can be of any use reviewing the others. 2008-04-28 09:44 willn, I am going to ping flips for the review, but thanks a lot all the same 2008-04-28 09:46 Sure thing 2008-04-28 09:47 zbuild: zumastor b0.8.0 r1601 build success 2008-04-28 09:47 zbuild: [PPA zumastor-team] Accepted: zumastor 0.8.0-r1601 (source) 2008-04-28 09:47 zbuild: [PPA zumastor-team] Accepted: ddsnap 0.8.0-r1601 (source) 2008-04-28 09:57 zbuild: zumastor b0.8.0 r1601 install success 2008-04-28 10:05 -!- jiayingz_(~jiayingz@207.47.98.129.static.nextweb.net) has joined #zumastor 2008-04-28 10:10 jiayingz_, good morning 2008-04-28 10:11 flips: Can you review the patches in issues #46, #113, #114, and #121 2008-04-28 10:11 doing now 2008-04-28 10:11 flips, could you review the three patches that are mentioned in the email i send to u 2008-04-28 10:11 flips, thanks! 2008-04-28 10:14 jiaying, could you use the form 2.patch instead of patch2 when you update a patch, please? patch2 confuses the browser 2008-04-28 10:14 so it takes even more steps to see the patch 2008-04-28 10:15 it's pretty bad as it is 2008-04-28 10:17 flips, i will name patches as you suggested in the future 2008-04-28 10:19 jiayingz_, we don't need to name files like ddsnap-xxx, because these files are already in the ddsnap directory 2008-04-28 10:19 we known they are for ddsnap 2008-04-28 10:19 so somelthing simple like ugprade.c 2008-04-28 10:20 first comment: I don't thing this effort to upgrade the superblock is a good thing to do at all 2008-04-28 10:20 flips, the program will be installed in /sbin 2008-04-28 10:20 go back and revisit why we're trying to do this 2008-04-28 10:20 jiayingz_, good point 2008-04-28 10:20 so let's think about the second point 2008-04-28 10:20 so i think it is better to name it so that people know where the program is used for 2008-04-28 10:20 why are we introducting this cvruft 2008-04-28 10:20 jiqyingz_, yes if it goes in sbin it has to be named like that 2008-04-28 10:20 flips, could u post ur comments to the issue tracker? 2008-04-28 10:20 ddsnap-upgrade for example 2008-04-28 10:21 or email 2008-04-28 10:21 email please 2008-04-28 10:21 issue tracker sucks too much 2008-04-28 10:21 beyond listing issues it falls down on most of the jobs it is supposed to do 2008-04-28 10:21 like enable people to work together effectively 2008-04-28 10:23 i think it is easy to track bug fixes with issue tracker 2008-04-28 10:24 fsck, firefox treats .patch as a binary that it won't open 2008-04-28 10:24 it is hard to track patches with emails after a while 2008-04-28 10:24 jiayingz_, issue tracker is no good for reviewing patches 2008-04-28 10:24 if it would post patches inline and accept replies from the list then it would be ok 2008-04-28 10:25 not good, just ok 2008-04-28 11:01 jiayingz_, use foo.patch.txt in email then, it'll make it easier to open. 2008-04-28 11:01 dkegel, ok 2008-04-28 11:01 flips, jiaying, tomorrow can you come in for the OKR meeting, 10:15? 2008-04-28 11:02 see you then 2008-04-28 11:02 Better make it 10AM just in case, the schedule has been moving around a bit. 2008-04-28 11:02 sure 2008-04-28 11:03 Thanks. Is it legal to use rocket shoes that early in the morning? 2008-04-28 11:03 With the sonic boom and all 2008-04-28 11:06 https://wiki.ubuntu.com/UbuntuOpenWeek has an 'intro to the ubuntu server team' session, wonder if it'd be useful... 2008-04-28 11:10 -!- MaZe(~MaZe@216-239-45-4.google.com) has joined #zumastor 2008-04-28 11:19 03jiahotcake * r1602 10/trunk/zumastor/bin/zumastor: 2008-04-28 11:19 Fix issue 114: snapshot kept for multiple rotations is umounted 2008-04-28 11:19 when it is removed from one rotation. 2008-04-28 11:32 dkegel: I'd say it would be useful 2008-04-28 11:37 zbuild: [PPA zumastor-team] Accepted: zumastor 0.8.0-r1602 (source) 2008-04-28 11:37 zbuild: [PPA zumastor-team] Accepted: ddsnap 0.8.0-r1602 (source) 2008-04-28 11:38 zbuild: zumastor b0.8.0 r1601 test success 2008-04-28 11:48 zbuild: zumastor b0.8.0 r1602 build success 2008-04-28 11:53 zbuild: zumastor b0.8.0 r1602 install success 2008-04-28 12:03 hi flips 2008-04-28 12:28 Looks like the replication tests are working :) 2008-04-28 12:28 Yay... are the smoke tests fully functional, then? Should I try them on a random fresh system? 2008-04-28 12:30 I've still not run the smoke test harness properly, though I've been testing on hardy not dapper. 2008-04-28 12:30 If you could give it a try on dapper, it should get alot farther than it used to 2008-04-28 12:37 how about gutsy? 2008-04-28 12:41 Not sure. 2008-04-28 12:42 any feedback is useful feedback, unless its just that you couldnt get the shell script to run ^-^ 2008-04-28 12:58 mmm, haribo. 2008-04-28 13:28 zbuild: zumastor b0.8.0 r1602 test success 2008-04-28 13:33 willn, i tried smoke test on dapper, but it failed during 'ddsnap initialize' 2008-04-28 13:33 the error is "main: Could not open snapshot store /dev/ubdc: No such device or address" 2008-04-28 13:36 Ok, I've got two dapper hosts coming up for testing, i'll work out the kinds. 2008-04-28 13:36 kinks* 2008-04-28 13:38 I'm also poking the chained-replication cbtb test, since I know that works 2008-04-28 14:46 -!- phillips_(~phillips@207.47.98.129.static.nextweb.net) has joined #zumastor 2008-04-28 15:55 03williamanowak * r1603 10/trunk/cbtb/uml/ (test-zuma-uml-dapper-i386.sh debootstrap-dapper-i386.sh): These two scripts are useless. Not called by anything afaict. 2008-04-28 16:03 zbuild: [PPA zumastor-team] Accepted: ddsnap 0.8.0-r1603 (source) 2008-04-28 16:18 zbuild: zumastor b0.8.0 r1603 build success 2008-04-28 16:18 zbuild: [PPA zumastor-team] Accepted: zumastor 0.8.0-r1603 (source) 2008-04-28 16:28 zbuild: zumastor b0.8.0 r1603 install success 2008-04-28 16:34 -!- jiayingz_(~jiayingz@207.47.98.129.static.nextweb.net) has joined #zumastor 2008-04-28 17:18 I think i've got the uml stuff fixed, doing another test. 2008-04-28 17:32 03williamanowak * r1604 10/trunk/cbtb/uml/ (setup.sh test-zuma-uml.sh): 2008-04-28 17:32 This commit should make UML testing work for the world. 2008-04-28 17:32 - Instead of telling people to fix permissions, just do it 2008-04-28 17:32 - Actually read the right variable for device sizing 2008-04-28 17:32 ACTION does the passing tests dance 2008-04-28 17:44 zbuild: [PPA zumastor-team] Accepted: zumastor 0.8.0-r1604 (source) 2008-04-28 17:44 zbuild: [PPA zumastor-team] Accepted: ddsnap 0.8.0-r1604 (source) 2008-04-28 17:49 zbuild: zumastor b0.8.0 r1604 build success 2008-04-28 17:59 zbuild: zumastor b0.8.0 r1603 test success 2008-04-28 17:59 zbuild: zumastor b0.8.0 r1604 install success 2008-04-28 18:01 -!- jiayingz_(~jiayingz@207.47.98.129.static.nextweb.net) has joined #zumastor 2008-04-28 18:29 03williamanowak * r1605 10/trunk/cbtb/ (15 files in 3 dirs): 2008-04-28 18:29 Many CBTB cleanup things: 2008-04-28 18:29 - Make all 2 and 3 node tests use the new device naming format 2008-04-28 18:29 - Update UML runners to use bash for sure. (TODO: Fix later) 2008-04-28 18:29 - Small whitespace cleaup 2008-04-28 18:34 haha! It works on a vanilla gutsy install. 2008-04-28 18:44 zbuild: zumastor b0.8.0 r1605 build success 2008-04-28 18:44 zbuild: [PPA zumastor-team] Accepted: zumastor 0.8.0-r1605 (source) 2008-04-28 18:44 zbuild: [PPA zumastor-team] Accepted: ddsnap 0.8.0-r1605 (source) 2008-04-28 18:54 zbuild: zumastor b0.8.0 r1605 install success 2008-04-28 19:39 zbuild: zumastor b0.8.0 r1604 test success 2008-04-28 21:17 03williamanowak * r1606 10/trunk/cbtb/tests/2/replication-snapshots-zumastor.sh: This test should no longer fail, since issue 26 was resolved. 2008-04-28 21:20 zbuild: zumastor b0.8.0 r1605 test success 2008-04-28 21:27 -!- MaZe(~MaZe@c-24-6-86-168.hsd1.ca.comcast.net) has joined #zumastor 2008-04-28 21:30 zbuild: [PPA zumastor-team] Accepted: zumastor 0.8.0-r1606 (source) 2008-04-28 21:30 zbuild: [PPA zumastor-team] Accepted: ddsnap 0.8.0-r1606 (source) 2008-04-28 21:40 zbuild: zumastor b0.8.0 r1606 build success 2008-04-28 21:50 zbuild: zumastor b0.8.0 r1606 install success 2008-04-28 22:18 http://lenz.homelinux.org/archives/182-Zumastor-as-an-alternative-for-LVMDRBD.html 2008-04-28 22:18 interesting link 2008-04-28 22:19 woo, about us! 2008-04-28 22:19 no one responded 2008-04-28 22:19 should invite to the mailing list 2008-04-28 22:19 yes 2008-04-28 22:20 picked that up by a google alert? 2008-04-28 22:22 ddsnap and zumastor actually need to be "advertisedf" separately 2008-04-28 22:22 referer logs 2008-04-28 22:22 see how the reader in this case inferred that zumastor was something like drbd 2008-04-28 22:23 well... ddsnap and drbd are somewhat alike 2008-04-28 22:23 i think the reader understood 2008-04-28 22:23 I agree 2008-04-28 22:23 it is an alternative to using drbd 2008-04-28 22:23 but he had to infer 2008-04-28 22:23 hm 2008-04-28 22:23 we should just state what is what 2008-04-28 22:23 well.. async drbd 2008-04-28 22:23 remote resemblance 2008-04-28 22:23 heh 2008-04-28 22:24 depends who you ask 2008-04-28 22:24 and what their application is ;) 2008-04-28 23:25 zbuild: zumastor b0.8.0 r1606 test success 2008-04-28 23:33 -!- mladjo(~mladjo@91.150.119.132) has joined #zumastor 2008-04-28 23:35 I just wanted to say Hi to all of you. My name is Mladen Djordjevic and I am student assigned to your organization 2008-04-28 23:35 fot this year's Google Summer of Code 2008-04-28 23:43 Hi Mladen :) 2008-04-28 23:44 Hi :) 2008-04-28 23:45 My task is first to make simple command line utility that will be able to list snapshots of a file/directory 2008-04-28 23:45 Coding begins on May 26th, but I was thinking to get familiar with community before that 2008-04-28 23:45 :) 2008-04-28 23:48 What do you think is the best way for installing zumastor, normally or in VM? 2008-04-28 23:49 ah cool, that is the task to add snapshot support to the gui eventually? 2008-04-28 23:49 yes 2008-04-28 23:49 to konqueror/nautilis 2008-04-28 23:49 its about the same either way, vm or real machine 2008-04-28 23:49 as long as you leave some disk space to play with 2008-04-28 23:50 I was thinking in VM just to be safe 2008-04-28 23:50 the kernel images we build are tested to work with vm 2008-04-28 23:50 my current kernel 2008-04-28 23:51 is 2.6.24 2008-04-28 23:51 i think that latest zumastor kernel is something like 2.6.22 2008-04-28 23:51 we have kernel patches for 2.6.24.2 2008-04-28 23:51 great 2008-04-28 23:51 ok 2008-04-28 23:51 no problem then 2008-04-28 23:51 thats for top of tree 2008-04-28 23:52 we haven't done a release on that kernel yet 2008-04-28 23:52 but it seems to be stable 2008-04-28 23:52 I can install 2.6.22 zumastor kernel then separately irc.oftc.net #zumastor log beginning Tue Apr 29 00:00:01 PDT 2008 2008-04-29 00:34 -!- mladjo(~mladjo@91.150.119.132) has left #zumastor 2008-04-29 05:39 -!- pgquiles(~pgquiles@105.Red-88-0-158.dynamicIP.rima-tde.net) has joined #zumastor 2008-04-29 08:00 -!- pgquiles_(~pgquiles@195.Red-83-39-142.dynamicIP.rima-tde.net) has joined #zumastor 2008-04-29 08:15 the chained replication test is still borked.. 2008-04-29 08:15 hmm 2008-04-29 08:16 -!- tim_vimm(~Tim@cpe-76-90-98-247.socal.res.rr.com) has joined #zumastor 2008-04-29 09:52 03jiahotcake * r1607 10/trunk/ddsnap/ddsnapd.c: 2008-04-29 09:52 Fix issue 113: ddsnap auto_delete code skips zero-usecount squashed snapshot. 2008-04-29 09:52 Add another function find_unused() that finds a zero-usecount snapshot for 2008-04-29 09:52 auto deleting when we reach the 64 maximum snapshot limit. 2008-04-29 09:56 -!- jiayingz_(~jiayingz@207.47.98.129.static.nextweb.net) has joined #zumastor 2008-04-29 10:01 zbuild: [PPA zumastor-team] Accepted: zumastor 0.8.0-r1607 (source) 2008-04-29 10:01 zbuild: [PPA zumastor-team] Accepted: ddsnap 0.8.0-r1607 (source) 2008-04-29 10:06 zbuild: zumastor b0.8.0 r1607 build success 2008-04-29 10:21 zbuild: zumastor b0.8.0 r1607 install success 2008-04-29 10:27 -!- cbsmith(~chatzilla@207.47.98.129.static.nextweb.net) has joined #zumastor 2008-04-29 10:28 -!- phoenix24(orqvj@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-29 11:52 zbuild: zumastor b0.8.0 r1607 test success 2008-04-29 12:04 hi willn 2008-04-29 12:04 hi jiayingz 2008-04-29 12:05 do u know how the sizes of origin device and snapshot device are set in our uml tests? 2008-04-29 12:05 i set DEV1SIZE=8 and DEV2SIZE=4 2008-04-29 12:06 but the size of the second device does not seem to be 4M 2008-04-29 12:07 which test? 2008-04-29 12:08 i tried snapshot-ddsnap.sh test 2008-04-29 12:09 oh jeez. 2008-04-29 12:09 I see the problem 2008-04-29 12:09 it seems it is always set to 64M 2008-04-29 12:11 can you svn up and try agani 2008-04-29 12:11 ok 2008-04-29 12:12 03williamanowak * r1608 10/trunk/cbtb/uml/ (4 files): 2008-04-29 12:12 UML Cleanup 2008-04-29 12:12 - Remove build-i386, it is not used 2008-04-29 12:12 - Make sure smoke-etch and build-etch fail if packages are not built 2008-04-29 12:12 - Fix the device size detector logic to actually look at more than dev1 2008-04-29 12:17 willn, the device size is right now. thx! 2008-04-29 12:24 03jiahotcake * r1609 10/trunk/cbtb/tests/1/ddsnap_autodelete.sh: Add a cbtb test to test ddsnap snapshot squashing and auto delete. 2008-04-29 12:27 zbuild: [PPA zumastor-team] Accepted: zumastor 0.8.0-r1608 (source) 2008-04-29 12:27 zbuild: [PPA zumastor-team] Accepted: ddsnap 0.8.0-r1608 (source) 2008-04-29 12:32 zbuild: zumastor b0.8.0 r1608 build success 2008-04-29 12:37 zbuild: [PPA zumastor-team] Accepted: zumastor 0.8.0-r1609 (source) 2008-04-29 12:37 zbuild: [PPA zumastor-team] Accepted: ddsnap 0.8.0-r1609 (source) 2008-04-29 12:42 zbuild: zumastor b0.8.0 r1608 install success 2008-04-29 12:47 zbuild: zumastor b0.8.0 r1609 build success 2008-04-29 12:52 zbuild: zumastor b0.8.0 r1609 install success 2008-04-29 13:09 jiayingz: great. let me know of any other issues 2008-04-29 14:00 hi flips 2008-04-29 14:00 hi jiaying 2008-04-29 14:02 big commit notification coming 2008-04-29 14:02 03williamanowak * r1610 10/trunk/ (5 files in 4 dirs): (log message trimmed) 2008-04-29 14:02 This change will unify version output for ddsnap and zumastor 2008-04-29 14:02 ddsnap 0.8.0-r1609m built on Tue Apr 29 13:56:49 PDT 2008 by willn@lackey 2008-04-29 14:02 and 2008-04-29 14:02 zumastor 0.8.0-r1609m built on Tue Apr 29 13:56:49 PDT 2008 by willn@lackey 2008-04-29 14:02 - Update debian rules clean action for zumastor 2008-04-29 14:02 - Update /etc/zumastor/version target in zumastor Makefile 2008-04-29 14:02 what timing I have 2008-04-29 14:02 suhweet 2008-04-29 14:03 it's a nice cleanup, that call to svnversion in the makefile was causing me serious eye hurt 2008-04-29 14:04 You just have a SVN grudge, I understand 2008-04-29 14:04 ;) 2008-04-29 14:06 ACTION goes back to breaking amd64 2008-04-29 14:06 flips, could you review the patches for issue 121 and 46 2008-04-29 14:06 OK 2008-04-29 14:07 thx! 2008-04-29 14:08 jiayingz, is it ddsnap-sb-upgrade.patch2 for #46? 2008-04-29 14:09 yes 2008-04-29 14:10 sorry I didn't already do it 2008-04-29 14:11 actually I renamed the patch as ddsnap-sb-upgrade.2.patch and send that to u by email 2008-04-29 14:18 zbuild: zumastor b0.8.0 r1610 build failure 1 2008-04-29 14:18 zbuild: [PPA zumastor-team] Accepted: zumastor 0.8.0-r1610 (source) 2008-04-29 14:18 zbuild: zumastor b0.8.0 r1608 test success 2008-04-29 14:18 zbuild: [PPA zumastor-team] Accepted: ddsnap 0.8.0-r1610 (source) 2008-04-29 14:22 03williamanowak * r1611 10/trunk/ (cbtb/host-scripts/dapper-build.sh buildcurrentsrc.sh): 2008-04-29 14:22 Fix a broken build, svnrev -> revision 2008-04-29 14:22 also fix the src deb builder 2008-04-29 14:38 zbuild: zumastor b0.8.0 r1611 build success 2008-04-29 14:38 zbuild: [PPA zumastor-team] Accepted: zumastor 0.8.0-r1611 (source) 2008-04-29 14:38 zbuild: [PPA zumastor-team] Accepted: ddsnap 0.8.0-r1611 (source) 2008-04-29 14:53 zbuild: zumastor b0.8.0 r1611 install success 2008-04-29 15:17 http://www.ccs.neu.edu/home/wan/zumastor-svn-stats/ 2008-04-29 15:24 -!- dkegel(~chatzilla@207.47.98.129.static.nextweb.net) has joined #zumastor 2008-04-29 15:25 those svn stats are sweet. It'd be nice to have that built in to code.google.com... 2008-04-29 15:26 I want xml/rss feeds first 2008-04-29 15:27 ping flips 2008-04-29 15:44 hi jiayingz 2008-04-29 15:46 have u looked at the patch for issue 121? 2008-04-29 15:53 jiayingz: did you see the failure in rev 1609? 2008-04-29 15:53 zbuild: zumastor b0.8.0 r1609 test failure 255 2008-04-29 15:54 heh. 2008-04-29 15:58 willn, i added that test. it passed the uml test. 2008-04-29 15:58 there is a problem with 'pkill ddsnap' 2008-04-29 15:58 Yea. 2008-04-29 15:59 hmm, wonder why 2008-04-29 15:59 i saw u use 'killall ddsnap' 2008-04-29 16:00 but that does not really kill 'ddsnap server' and 'ddsnap agent', because we changed their names to 'ddsnap_server' and 'ddsnap_agent' 2008-04-29 16:05 ick. 2008-04-29 16:06 http://www.ccs.neu.edu/home/wan/zumastor-svn-stats-proper/ 2008-04-29 16:06 dkegel: ^ 2008-04-29 16:28 jiayingz, how nice to only have to read a one line patch ;-) 2008-04-29 16:30 flips, :) 2008-04-29 16:33 03williamanowak * r1612 10/trunk/cbtb/apt-repository: This is not used anywhere 2008-04-29 16:38 03jiahotcake * r1613 10/trunk/ddsnap/ (7 files in 2 dirs): 2008-04-29 16:38 Add a helper program, ddsnap-sb, that implements ddsnap superblock upgrade. 2008-04-29 16:38 Every time when the ondisk superblock changes, we should add a new upgrade 2008-04-29 16:38 function in ddsnap-sb.c that reads in the old ondisk structure, plugs the 2008-04-29 16:38 information to the new structure, and writes the new superblock to the disk. 2008-04-29 16:39 See issue 46. 2008-04-29 16:45 03williamanowak * r1614 10/trunk/ddsnap/debian/ (ddsnap.manpages rules): 2008-04-29 16:45 Add ddsnap-sb to debian manpages list and binary install list within our 2008-04-29 16:45 packages. 2008-04-29 16:45 jiayingz: that last commit was missing, I forgot to look at the patch you committed and comment on that. 2008-04-29 16:46 03jiahotcake * r1615 10/trunk/ddsnap/ddsnapd.c: 2008-04-29 16:46 Fix issue 121: Replication may stop when snapshots are squashed on downstream. 2008-04-29 16:46 Set the transient usecount of a squashed snapshot to zero. 2008-04-29 17:03 zbuild: zumastor b0.8.0 r1612 build success 2008-04-29 17:09 zbuild: zumastor b0.8.0 r1615 build success 2008-04-29 17:09 zbuild: zumastor b0.8.0 r1612 install success 2008-04-29 17:09 zbuild: zumastor b0.8.0 r1614 build success 2008-04-29 17:22 03jiahotcake * r1616 10/trunk/cbtb/tests/1/ddsnap_autodelete.sh: 2008-04-29 17:22 Try "pkill -f 'ddsnap agent'" instead of "pkill ddsnap" 2008-04-29 17:22 in the cbtb ddsnap_autodelete.sh test. 2008-04-29 17:24 zbuild: zumastor b0.8.0 r1614 install success 2008-04-29 17:34 zbuild: zumastor b0.8.0 r1615 install success 2008-04-29 17:34 zbuild: [PPA zumastor-team] Accepted: zumastor 0.8.0-r1615 (source) 2008-04-29 17:34 zbuild: [PPA zumastor-team] Accepted: ddsnap 0.8.0-r1615 (source) 2008-04-29 17:44 zbuild: zumastor b0.8.0 r1616 build success 2008-04-29 17:44 zbuild: zumastor b0.8.0 r1611 test failure 1 2008-04-29 17:44 zbuild: [PPA zumastor-team] Accepted: zumastor 0.8.0-r1616 (source) 2008-04-29 17:44 zbuild: [PPA zumastor-team] Accepted: ddsnap 0.8.0-r1616 (source) 2008-04-29 17:49 -!- jiayingz_(~jiayingz@207.47.98.129.static.nextweb.net) has joined #zumastor 2008-04-29 17:50 zbuild: zumastor b0.8.0 r1616 install success 2008-04-29 19:25 zbuild: zumastor b0.8.0 r1615 test failure 255 2008-04-29 20:48 -!- tim_vimm(~Tim@cpe-76-90-98-247.socal.res.rr.com) has joined #zumastor 2008-04-29 21:05 zbuild: zumastor b0.8.0 r1616 test success 2008-04-29 22:16 -!- phoenix24(gzxuw@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-29 22:22 -!- phoenix24(rzu@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-29 22:58 -!- tim_vimm(~Tim@cpe-76-90-98-247.socal.res.rr.com) has joined #zumastor 2008-04-29 23:25 -!- phoenix24(mow@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-29 23:49 03Daniel.Raymond.Phillips * r1617 10/www/graphs/ (bitmap_dell2950.jpg bitmap_hpx4100.jpg): Add bitmap optimization graphs irc.oftc.net #zumastor log beginning Wed Apr 30 00:00:01 PDT 2008 2008-04-30 00:00 zbuild: [PPA zumastor-team] Accepted: zumastor 0.8.0-r1617 (source) 2008-04-30 00:00 zbuild: [PPA zumastor-team] Accepted: ddsnap 0.8.0-r1617 (source) 2008-04-30 00:43 -!- phoenix24(bce@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-30 00:45 googlegroups really sucks ass as a mailing list 2008-04-30 00:45 sorry, mothership, but it's true 2008-04-30 00:45 trying to post a link to a post in the archive is eye gougingly horrible 2008-04-30 02:25 ACTION fires a the "long road to optimal" post at lkml 2008-04-30 02:25 *yawn* 2008-04-30 02:54 -!- flips(~phillips@phunq.net) has joined #zumastor 2008-04-30 05:00 -!- pgquiles__(~pgquiles@62.43.226.52.static.user.ono.com) has joined #zumastor 2008-04-30 05:40 -!- pgquiles_(~pgquiles@195.Red-83-39-142.dynamicIP.rima-tde.net) has joined #zumastor 2008-04-30 05:56 zissues: Issue 124 in zumastor: Ubuntu: missing packages 2008-04-30 07:54 -!- tim_vimm(~Tim@cpe-76-90-98-247.socal.res.rr.com) has joined #zumastor 2008-04-30 09:51 03jiahotcake * r1618 10/trunk/cbtb/tests/1/zumastor_autodelete.sh: 2008-04-30 09:51 Add a cbtb test to verify that zumastor handles snapshot squashing/deleting 2008-04-30 09:51 correctly. 2008-04-30 10:06 zbuild: [PPA zumastor-team] Accepted: zumastor 0.8.0-r1618 (source) 2008-04-30 10:06 zbuild: [PPA zumastor-team] Accepted: ddsnap 0.8.0-r1618 (source) 2008-04-30 10:12 -!- phoenix24(phtnjih@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-30 10:21 zbuild: zumastor b0.8.0 r1618 install success 2008-04-30 10:21 zbuild: zumastor b0.8.0 r1618 build success 2008-04-30 11:05 -!- phoenix24_(enpnt@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-30 11:29 -!- MaZe(~MaZe@216-239-45-4.google.com) has joined #zumastor 2008-04-30 12:02 zissues: Issue 29 in zumastor: Unit test framework 2008-04-30 12:02 zissues: Issue 97 in zumastor: Support AIO 2008-04-30 12:05 -!- phoenix24(ofigdt@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-30 12:06 zbuild: zumastor b0.8.0 r1618 test success 2008-04-30 12:29 -!- tim_vimm(~Tim@cpe-76-90-98-247.socal.res.rr.com) has joined #zumastor 2008-04-30 13:27 zbuild: [PPA zumastor-team] Accepted: linux-meta 2.6.24.16.18~ppa5 (source) 2008-04-30 13:50 03williamanowak * r1619 10/ (3 files in 3 dirs): 2008-04-30 13:50 Per a chat with Shapor, i'm going to list myself as the maintainer for the 2008-04-30 13:50 debs. 2008-04-30 13:52 zbuild: [PPA zumastor-team] Accepted: linux-ubuntu-modules-2.6.24 2.6.24-16.23~ppa5 (source) 2008-04-30 14:04 03williamanowak * r1620 10/trunk/ (7 files in 5 dirs): Get rid of 2.6.21.1 cruft 2008-04-30 14:07 zbuild: [PPA zumastor-team] Accepted: zumastor 0.8.0-r1619 (source) 2008-04-30 14:07 zbuild: [PPA zumastor-team] Accepted: ddsnap 0.8.0-r1619 (source) 2008-04-30 14:22 zbuild: [PPA zumastor-team] Accepted: ddsnap 0.8.0-r1620 (source) 2008-04-30 14:22 zbuild: zumastor b0.8.0 r1619 install success 2008-04-30 14:22 zbuild: [PPA zumastor-team] Accepted: zumastor 0.8.0-r1620 (source) 2008-04-30 14:22 zbuild: zumastor b0.8.0 r1619 build success 2008-04-30 15:02 zissues: Issue 125 in zumastor: ddsnap server fails to set PF_MEMALLOC with 2.4.24.2 2008-04-30 15:31 03jiahotcake * r1621 10/trunk/ddsnap/ddsnapd.c: 2008-04-30 15:31 Change the values of PR_SET_LESS_THROTTLE and PR_SET_MEMALLOC in ddsnapd.c 2008-04-30 15:31 to be consistent with the current 2.6.24.2 kernel. 2008-04-30 15:43 zbuild: [PPA zumastor-team] Accepted: zumastor 0.8.0-r1621 (source) 2008-04-30 15:43 zbuild: [PPA zumastor-team] Accepted: ddsnap 0.8.0-r1621 (source) 2008-04-30 15:53 03williamanowak * r1622 10/trunk/cbtb/uml/ (debootstrap-etch-i386.sh smoke-etch.sh): Make sure scripts are run with bash, fail if no ssh key found. 2008-04-30 16:08 zbuild: zumastor b0.8.0 r1619 test success 2008-04-30 16:08 zbuild: [PPA zumastor-team] Accepted: zumastor 0.8.0-r1622 (source) 2008-04-30 16:08 zbuild: [PPA zumastor-team] Accepted: ddsnap 0.8.0-r1622 (source) 2008-04-30 16:12 -!- tim_vimm(~Tim@cpe-76-90-98-247.socal.res.rr.com) has joined #zumastor 2008-04-30 16:35 03williamanowak * r1623 10/trunk/cbtb/ (2 files in 2 dirs): 2008-04-30 16:35 -Make the UML tester fail on hardy, due to libc6 errors 2008-04-30 16:35 -Force xfs installation for cbtb/tests/1/snapshot-zumastor-xfs-2045G.sh 2008-04-30 18:07 03williamanowak * r1624 10/trunk/cbtb/uml/ (8 files): 2008-04-30 18:07 Refactor everything. 2008-04-30 18:07 Works on hardy, amd64 and probably other things too. 2008-04-30 18:43 zbuild: [PPA zumastor-team] Accepted: zumastor 0.8.0-r1624 (source) 2008-04-30 18:43 zbuild: [PPA zumastor-team] Accepted: ddsnap 0.8.0-r1624 (source) 2008-04-30 18:43 zbuild: [PPA zumastor-team] Accepted: linux-restricted-modules-2.6.24 2.6.24.12-16.34~ppa5 (source) 2008-04-30 18:43 zbuild: [PPA zumastor-team] Accepted: zumastor 0.8.0-r1623 (source) 2008-04-30 18:43 zbuild: [PPA zumastor-team] Accepted: ddsnap 0.8.0-r1623 (source) 2008-04-30 19:40 so, uh, uml is totally rockin' now. amd64, i386, ubuntu, debian, hardy, dapper, gutsy, all should work. 2008-04-30 19:58 -!- tim_vimm(~Tim@cpe-76-90-98-247.socal.res.rr.com) has joined #zumastor 2008-04-30 20:57 pgquiles: I think the ppa is all kosher now 2008-04-30 21:08 Yep 2008-04-30 21:08 I'm running a zumastor enabled kernel on my laptop with nvidia drivers and whatnot 2008-04-30 21:12 and hey, zumastor works too 2008-04-30 22:44 -!- tim_vimm(~Tim@cpe-76-90-98-247.socal.res.rr.com) has joined #zumastor 2008-04-30 23:21 -!- phoenix24(itu@h1119083.serverkompetenz.net) has joined #zumastor 2008-04-30 23:32 -!- MaZe(~MaZe@c-24-6-86-168.hsd1.ca.comcast.net) has joined #zumastor