irc.oftc.net #zumastor log beginning Wed Aug 1 00:00:01 PDT 2007 2007-08-01 14:04 drat, last night's email configuration "adventures" got me unsubscribed from lkml 2007-08-01 14:08 shapor, there? 2007-08-01 14:39 flipz: yeah 2007-08-01 14:39 hi 2007-08-01 14:39 well I'm resubscribed to lkml 2007-08-01 14:40 you should setup a backup mx 2007-08-01 14:40 since lkml is so picky 2007-08-01 14:40 exim's default queue management and lkml's mail policy combine as a kooky cocktail of timewasting idiocy 2007-08-01 14:40 yeah the greylisting issue i imagine 2007-08-01 14:41 not to mention the entire idiotic history of mta's... there that should get it off my chest 2007-08-01 14:46 http://www.exim.org/exim-html-current/doc/html/spec_html/ch07.html#SECID57 2007-08-01 14:56 ooh, thanks 2007-08-01 16:18 -!- cbsmith(~user@207.47.98.129.static.nextweb.net) has joined #zumastor 2007-08-01 16:59 I'm on the verge of ordering this little brick: http://www.linuxdevices.com/news/NS5551137361.html 2007-08-01 16:59 but I can't find a single hit on the web to somebody who has one, much less a review 2007-08-01 17:00 I'm not sure I want to be the alpha test 2007-08-01 17:01 I saw an $99 unit just recently.... 2007-08-01 17:01 the question is, do I feel lucky? 2007-08-01 17:02 the difference most probably is, the brick above is a real pc 2007-08-01 17:02 you were likely looking at an arm-based applicance 2007-08-01 17:02 which requires considerable effort to customize 2007-08-01 17:03 flipz: it was designed to be customized... let me see where it was again... 2007-08-01 17:03 color me interested 2007-08-01 17:04 I remember thinking it might be able to fly as a MythTV FE for standard def TV at least. 2007-08-01 17:05 that would be impressive 2007-08-01 17:05 flipz: It's a Via C7 eden @1.2GHz w/512MB of RAM. 2007-08-01 17:05 url? 2007-08-01 17:05 http://www.zonbu.com/home/index.htm 2007-08-01 17:05 Doesn't include keyboard or monitor, but you know. 2007-08-01 17:05 Uses Gentoo. 2007-08-01 17:06 I'm sure that could be fixed ;) 2007-08-01 17:06 I seem to recall by the way 2007-08-01 17:06 it was the gentoo that got my attention 2007-08-01 17:06 flipz: feature, not bug ;-) 2007-08-01 17:06 flipz: particularly handy since it is an ideal device for using NFS. ;-) 2007-08-01 17:06 I was going to tell you about my gentoo experience 2007-08-01 17:07 flipz: All candies and roses I imagine. ;-) 2007-08-01 17:07 9 months as my workstation, the same one that is now running this xchat window and my web site 2007-08-01 17:07 and now running under debian etch 2007-08-01 17:07 I got tired of debugging libtool errors in gentoo 2007-08-01 17:08 and later, after I converted back to debian sid, I got tired of sid breaking things like my video and leaving them that way for months 2007-08-01 17:08 so downgrade to etch, and now I have an acceptable end user experience 2007-08-01 17:08 kind of novel for me 2007-08-01 17:09 I still have gentoo on one partition and sid on another, but they are unlikely to see the light of day again on this machine so I may recover the bits 2007-08-01 17:09 other machines at home are a difference story 2007-08-01 17:10 sid is fine if you have an efficient way of backing out an entire upgrade attempt 2007-08-01 17:10 flipz: It's funny, I almost never had to debug libtool errors. 2007-08-01 17:10 flipz: You mean like zumastor? ;-) 2007-08-01 17:10 I had to debug a lot of mis-installed libraries 2007-08-01 17:10 like zumastor, a year from now 2007-08-01 17:11 currently device mapper prevents it from being actually nice on the desktop 2007-08-01 17:11 Weird. I almost never had problems like that. My usual problems were more along the lines of, "someone broke the build in a hideous way... wait a day or two for the fix and then rebuild... in the meantime use the old version". 2007-08-01 17:11 possibly, you weren't right on the edge like I was 2007-08-01 17:12 I needed the very latest xorg to run my embedded intel graphics 2007-08-01 17:12 flipz: Were you doing ~x86 all the time? 2007-08-01 17:12 that involved a bunch of masking 2007-08-01 17:12 no 2007-08-01 17:12 always source 2007-08-01 17:12 except for firefox I think 2007-08-01 17:13 I didn't dislike gentoo, I just did not have the time to maintain it or wait for things to compile 2007-08-01 17:13 emerge beats the tail of apt 2007-08-01 17:13 flipz: yeah, masking was what I meant. I had to do that for X to avoid some nvidia bugs. Plus I did it for some gtk stuff to get the version of cairo I liked, plus C# stuff, plus Java stuff... I had like 50+ masks. Worked WAY better than trying to use 50+ packages from Sid on a testing system. 2007-08-01 17:14 flipz: The wait for things to compile thing is solvable. 2007-08-01 17:14 flipz: The debug thing used to not be a problem. It is becoming one though. 2007-08-01 17:14 flipz: Used to be I could report a bug and expect a fix in <24hrs, usually <2. Now it is different. 2007-08-01 17:14 when the intel stuff went into stable debian, the one reason I *had* to run gentoo went away 2007-08-01 17:15 I have run with xfree/xorg compiled from source on sid in the past, that is not pretty 2007-08-01 17:15 anyway I am glad somebody else likes gentoo, because I do 2007-08-01 17:15 I just wish it had a full suite of binary packages, that would rock 2007-08-01 17:16 then you could just get source for exactly what you need it for 2007-08-01 17:17 by the way, I need a hard disk, so can't use that $99 brick 2007-08-01 17:17 I'm actually working on setting up a server that would build binary packages for my UML coop. 2007-08-01 17:17 too bad 2007-08-01 17:17 flipz: NAS man. 2007-08-01 17:17 uml coop? 2007-08-01 17:18 this has to work as the only machine alive on my network, other than the router and dsl modem 2007-08-01 17:18 Yeah, I have >12 people sharing a machine using UML instances. 2007-08-01 17:18 so I would need to add a separate hard disk 2007-08-01 17:18 hard core 2007-08-01 17:18 flipz: Yeah, get one of those ATAoE drives. 2007-08-01 17:18 and we haven't been bugging you about uml issues??? 2007-08-01 17:19 flipz: We're probably going to shift to OpenVZ or VMware. I heavily favour OpenVZ. 2007-08-01 17:19 flipz: hehe. jiayangz has come to me with some stuff. 2007-08-01 17:19 is that KVM aka bochs? 2007-08-01 17:19 flipz: No, it's more like chroot on steroids. 2007-08-01 17:19 flipz: It turns out nobody but me was really interested in playing around with different kernels. ;-) 2007-08-01 17:21 looks like it is on the right track, and in line with kernel directions 2007-08-01 17:21 openvz that is 2007-08-01 17:21 it would be nice if full virtualization vs just namespaces was merely a per-vm mode switch 2007-08-01 17:22 yes. 2007-08-01 17:24 openvz looks very nice from 1,000 feet 2007-08-01 17:24 flipz: Yup. I wish the kernel guys would base more of their namespace work on it. It has some nice ideas, and it is functionally complete. 2007-08-01 17:25 well paul menage is one of the two people driving namespaces, he is a phone call away 2007-08-01 17:26 the other is christoph lameter of sgi, also highly approachable 2007-08-01 17:30 openvz seems to include some kernel hacks, good 2007-08-01 17:31 it's yum-orient, that raises a flag 2007-08-01 18:01 flipz: yes, it is yum-oriented, because their target market are hosting services, but there is a nice Gentoo HOWTO that shows you that you don't need it. 2007-08-01 18:06 So, one problem I just realized with Zonbu is the real price is $261.95. 2007-08-01 18:15 'cause you can't buy it without service. 2007-08-01 19:03 bleah 2007-08-01 19:09 flipz: yeah, I had a similar thought. irc.oftc.net #zumastor log beginning Thu Aug 2 00:00:01 PDT 2007 2007-08-02 10:51 -!- daniel_(~daniel@dsl081-085-152.lax1.dsl.speakeasy.net) has joined #zumastor 2007-08-02 12:19 ok, there we go, a mail to multiple recipients including lkml and linus successfully delivered 2007-08-02 12:19 I guess exim4 works 2007-08-02 12:20 not to mention dns courtesy of shaptech 2007-08-02 12:20 success :) 2007-08-02 12:20 stress went down by a step function 2007-08-02 12:21 http://lkml.org/lkml/2007/8/2/314 2007-08-02 12:21 just some random comment in a thread on gettimeofday by the way, not "the" patch 2007-08-02 12:22 yes, makes it all seem more real 2007-08-02 12:22 that was pretty darn fast of lkml.org 2007-08-02 12:22 compared to some mail archivers I could mention ;) 2007-08-02 12:23 hopefully it will take a long time for somebody to elucidate me, probably wrongly 2007-08-02 12:23 or couldn't mention for that matter ;) 2007-08-02 12:24 I can therefore get some work done in the mean time 2007-08-02 12:24 I sense the delicate touch of hpa aka peter anvin re lkml.org 2007-08-02 12:25 former principle engineer of transmeta 2007-08-02 12:26 (c) 2002-2007 Jasper Spaans 2007-08-02 12:51 -!- murb(~murbix@soapstone.yuri.org.uk) has joined #zumastor 2007-08-02 14:09 http://thread.gmane.org/gmane.linux.kernel/563301 <- a most wonderful thing, maybe 2007-08-02 14:24 johnpol@2ka.mipt.ru 2007-08-02 14:24 SMTP error from remote mail server after MAIL FROM:: 2007-08-02 14:24 host 2ka.mipt.ru [194.85.82.65]: 554 5.7.1 Looks like a dynamic address 2007-08-02 14:24 so he just dropped of the net work a while? 2007-08-02 14:39 no, his mail server doesn't like you 2007-08-02 14:39 its probably reversing the ip you connect from 2007-08-02 14:39 and seeing dsl081-085-152.lax1.dsl.speakeasy.net, which looks dynamic 2007-08-02 14:40 flipz: probably best to relay the email via something which doesn't look like a dialup IP. 2007-08-02 14:45 flipz: I know someone who used to have speakeasy and they set up reverse dns for him 2007-08-02 14:46 so you can probably request it to be changed to something like moonbase.phunq.net, and I can setup dns for you 2007-08-02 16:18 ok so maybe it is trival to do in shell: 2007-08-02 16:18 grep ^Rss /proc/1/smaps | awk '(a=a+$2) {} END {print a}' 2007-08-02 16:29 awk '{print $2}' /proc/1/statm 2007-08-02 16:29 even better 2007-08-02 16:30 lcut -d\ -f2 /proc/1/statm 2007-08-02 16:30 less characters 2007-08-02 16:30 oops s/^l// 2007-08-02 16:31 in pages rather than kb 2007-08-02 16:31 probably a lot less expensive on the kernel side too irc.oftc.net #zumastor log beginning Fri Aug 3 00:00:01 PDT 2007 2007-08-03 11:02 shapor, hmm you know what would be a nice addition to /proc? /proc/help/fieldnames/statm 2007-08-03 11:15 Use the Source, Luke. 2007-08-03 11:15 exactly, by amending to so it doesn't suck as much 2007-08-03 11:16 not that kernel config is now embedded in kernel by default, after fighting about whether that should be done for years 2007-08-03 11:16 note I mean 2007-08-03 11:17 fmayhar, quick, what do the fields in /proc/l/stam mean? 2007-08-03 11:17 2007-08-03 11:17 :_) 2007-08-03 11:19 2007-08-03 11:20 now are you a) always going to remember it is and not or b) have to use the source every time? 2007-08-03 11:23 I'll remember " ." If I care about the other stuff (which I typically don't), I'll look it up. 2007-08-03 11:24 incidentally, the way the in-kernel config finally got in was by ignoring all comments of the form "I don't need no steenking help and so nobody else should have it either" 2007-08-03 11:25 You'll note I left off the second half of that. :-) 2007-08-03 11:25 ;-) 2007-08-03 11:25 meanwhile, we have got buyin on lkml from our former biggest critic on the network memory deadlock prevention 2007-08-03 11:26 http://thread.gmane.org/gmane.linux.kernel/563301 2007-08-03 11:27 (cut n paste just broke again, I could not paste that url until I had raised the window, middle click in the window did not work) 2007-08-03 11:28 oh yeah nbd is crashtastic 2007-08-03 11:30 So can I commit that monitoring code? 2007-08-03 11:30 which, as bash or c code? 2007-08-03 11:30 C 2007-08-03 11:30 please no 2007-08-03 11:31 ACTION sighs. 2007-08-03 11:31 murb, seen it yourself I presume? 2007-08-03 11:34 flipz: yes but only with xen. 2007-08-03 11:34 that counts 2007-08-03 12:56 acme@ghostprotocols.net 2007-08-03 12:56 SMTP error from remote mail server after RCPT TO:: 2007-08-03 12:56 host mail5.zoneedit.com [216.55.181.47]: 504 : 2007-08-03 12:56 Helo command rejected: need fully-qualified hostname 2007-08-03 12:57 #hostname 2007-08-03 12:57 moonbase 2007-08-03 12:57 I wonder if I need to set hostname to moonbase.phunq.net? 2007-08-03 12:59 it sounds you need to do it 2007-08-03 13:23 ACTION tries to remember how to set hostname permanently on debian 2007-08-03 13:24 hmm, hostname calls kernel uname 2007-08-03 13:24 oh, and passes "moonbase" to it 2007-08-03 13:26 err, no, gets moonbase back from it 2007-08-03 13:26 kind of odd having kernel tell us the hostname 2007-08-03 13:28 kernel learns it from an earlier call to sethostname, so now I have to go trawling through init scripts 2007-08-03 13:29 init.d/hostname.sh... 2007-08-03 13:30 ...which tells me to write it in /etc/hostname, sheesh, I knew that 2007-08-03 13:53 flipz: edit /etc/hostname 2007-08-03 13:53 or with exim you can just set primary_hostname 2007-08-03 13:57 trying it to see if fqdn in /etc/hostname works out ok, for example it seems bash prompt is set up by default to only use the local hostname part 2007-08-03 13:57 which otherwise would be annoying 2007-08-03 13:57 that for the factoid about primary_hostname 2007-08-03 14:04 flipz you should set the hostname uses in your exim config 2007-08-03 14:05 you dont want it to use the system default hostname because your behind nat 2007-08-03 14:05 to satisfy even more picky servers you want to set it to whatever your public i reverses to 2007-08-03 14:05 public ip* 2007-08-03 14:06 aka what i have the mx record set to for your domain, dsl081-085-152.lax1.dsl.speakeasy.net 2007-08-03 14:17 I will ask speakeasy to set the reverse to moonbase.phunq.net 2007-08-03 14:17 once I get them to send me a new password ;) 2007-08-03 14:18 phillips law: all simple things become stupidly complex when you actually try to do them 2007-08-03 17:43 the .ru mails I'm failing to deliver are the direct cc's to Evgeniy 2007-08-03 17:44 2007-08-03 17:42:24 1IH7ho-0004kO-6U mailer.campus.mipt.ru [194.85.82.4] Connection refused 2007-08-03 19:58 -!- liewegas(~sage@dsl081-089-187.lax1.dsl.speakeasy.net) has joined #zumastor irc.oftc.net #zumastor log beginning Sat Aug 4 00:00:01 PDT 2007 2007-08-04 08:12 -!- flips(~phillips@207.47.98.129.static.nextweb.net) has joined #zumastor 2007-08-04 08:12 -!- jiayingz(~jiayingz@207.47.98.129.static.nextweb.net) has joined #zumastor irc.oftc.net #zumastor log beginning Sun Aug 5 00:00:01 PDT 2007 2007-08-05 07:53 -!- murb(~murbix@soapstone.yuri.org.uk) has joined #zumastor 2007-08-05 11:00 -!- mostafasicaf(~mostafa@196.202.32.195) has joined #zumastor 2007-08-05 11:48 -!- murb(~murbix@soapstone.yuri.org.uk) has joined #zumastor 2007-08-05 15:23 -!- murb(~murbix@soapstone.yuri.org.uk) has joined #zumastor 2007-08-05 15:23 -!- erwan_taf(~erwan@konilope.linuxeries.org) has joined #zumastor 2007-08-05 15:23 -!- juuva(juuva@peili.org) has joined #zumastor 2007-08-05 19:00 -!- murb(~murbix@soapstone.yuri.org.uk) has joined #zumastor 2007-08-05 19:00 -!- erwan_taf(~erwan@konilope.linuxeries.org) has joined #zumastor 2007-08-05 19:00 -!- juuva(juuva@peili.org) has joined #zumastor 2007-08-05 21:07 -!- murb(~murbix@soapstone.yuri.org.uk) has joined #zumastor 2007-08-05 21:07 -!- erwan_taf(~erwan@konilope.linuxeries.org) has joined #zumastor 2007-08-05 21:07 -!- juuva(juuva@peili.org) has joined #zumastor irc.oftc.net #zumastor log beginning Mon Aug 6 00:00:01 PDT 2007 2007-08-06 00:01 -!- erwan_taf(~erwan@konilope.linuxeries.org) has joined #zumastor 2007-08-06 09:37 actual size of peterz's patch set is ~2000 lines changed/added, ~6000 lines of patch total 2007-08-06 09:38 including email text 2007-08-06 10:12 seems like a lot more.. quite an action-packed patchset 2007-08-06 10:36 peterz has just posted an update 2007-08-06 10:37 the patch set is now broken up into groups of 10 or so 2007-08-06 10:43 http://lkml.org/lkml/2007/8/6/296 2007-08-06 12:11 looks like some good progress 2007-08-06 12:32 it's getting cleaner and better 2007-08-06 12:33 ACTION heads in for lunch 2007-08-06 12:54 -!- erwan_taf(~erwan@konilope.linuxeries.org) has joined #zumastor 2007-08-06 13:15 -!- erwan_taf(~erwan@konilope.linuxeries.org) has joined #zumastor 2007-08-06 13:38 -!- erwan_taf(~erwan@konilope.linuxeries.org) has joined #zumastor 2007-08-06 14:08 -!- flips(~phillips@207.47.98.129.static.nextweb.net) has joined #zumastor 2007-08-06 14:46 -!- cbsmith(~user@207.47.98.129.static.nextweb.net) has joined #zumastor irc.oftc.net #zumastor log beginning Tue Aug 7 00:00:03 PDT 2007 2007-08-07 06:23 -!- murb_(~murbix@soapstone.yuri.org.uk) has joined #zumastor 2007-08-07 12:46 -!- jdries3(~jdries3@72.14.228.89) has joined #zumastor 2007-08-07 13:40 -!- jdries3(~jdries3@72.14.228.89) has joined #zumastor 2007-08-07 14:22 -!- xman(~user@207.47.98.129.static.nextweb.net) has joined #zumastor 2007-08-07 14:51 What's the feeling on how to integrate 3rd party open source code? 2007-08-07 14:56 -!- jdries3(~jdries3@72.14.228.89) has joined #zumastor 2007-08-07 15:29 -!- jdries3(~jdries3@187.sub-75-193-94.myvzw.com) has joined #zumastor 2007-08-07 16:30 -!- jdries3(~jdries3@c-69-249-52-124.hsd1.nj.comcast.net) has joined #zumastor 2007-08-07 16:31 -!- jdries3_(~jdries3@72.14.224.1) has joined #zumastor 2007-08-07 19:43 -!- daniel__(~phlipz@adsl-63-202-13-187.dsl.snfc21.pacbell.net) has joined #zumastor irc.oftc.net #zumastor log beginning Wed Aug 8 00:00:02 PDT 2007 2007-08-08 12:01 -!- jdries3(~jdries3@72.14.228.89) has joined #zumastor 2007-08-08 14:06 -!- jdries3(~jdries3@72.14.228.89) has joined #zumastor 2007-08-08 14:41 -!- jdries3(~jdries3@72.14.228.89) has joined #zumastor 2007-08-08 15:07 -!- flipz(~daniel@dsl081-085-152.lax1.dsl.speakeasy.net) has joined #zumastor 2007-08-08 15:09 -!- jdries3(~jdries3@72.14.228.89) has joined #zumastor 2007-08-08 15:09 -!- flipz(~daniel@dsl081-085-152.lax1.dsl.speakeasy.net) has joined #zumastor 2007-08-08 15:26 -!- flipz(~daniel@dsl081-085-152.lax1.dsl.speakeasy.net) has joined #zumastor 2007-08-08 16:12 -!- jdries3(~jdries3@72.14.228.89) has joined #zumastor irc.oftc.net #zumastor log beginning Thu Aug 9 00:00:03 PDT 2007 2007-08-09 01:28 -!- daniel__(~phlipz@adsl-63-202-13-187.dsl.snfc21.pacbell.net) has joined #zumastor 2007-08-09 03:15 -!- erwan_taf(~erwan@konilope.linuxeries.org) has joined #zumastor 2007-08-09 12:00 -!- jdries3(~jdries3@72.14.228.89) has joined #zumastor 2007-08-09 12:50 -!- zumalog(~zumalog@yzf.shaptech.com) has joined #zumastor 2007-08-09 12:55 -!- shapor(~shapor@yzf.shaptech.com) has joined #zumastor 2007-08-09 17:38 -!- jdries3(~jdries3@72.14.228.1) has joined #zumastor 2007-08-09 21:52 -!- daniel__(~phlipz@adsl-63-202-13-187.dsl.snfc21.pacbell.net) has joined #zumastor irc.oftc.net #zumastor log beginning Fri Aug 10 00:00:03 PDT 2007 2007-08-10 11:00 -!- jdries3(~jdries3@72.14.228.89) has joined #zumastor 2007-08-10 11:12 -!- jdries3(~jdries3@72.14.228.89) has joined #zumastor 2007-08-10 12:11 -!- jdries3(~jdries3@72.14.228.89) has joined #zumastor 2007-08-10 14:10 -!- daniel__(~phlipz@216-239-45-4.google.com) has joined #zumastor 2007-08-10 16:43 -!- jdries3(~jdries3@c-69-249-52-124.hsd1.nj.comcast.net) has joined #zumastor 2007-08-10 19:52 -!- Tenchu(~ddmrulz@c-68-62-47-157.hsd1.mi.comcast.net) has joined #zumastor 2007-08-10 19:53 Oops 2007-08-10 19:53 -!- Tenchu(~ddmrulz@c-68-62-47-157.hsd1.mi.comcast.net) has left #zumastor 2007-08-10 23:39 -!- erwan_taf(~erwan@konilope.linuxeries.org) has joined #zumastor irc.oftc.net #zumastor log beginning Sat Aug 11 00:00:04 PDT 2007 irc.oftc.net #zumastor log beginning Sun Aug 12 00:00:03 PDT 2007 2007-08-12 11:18 -!- erwan_taf(~erwan@konilope.linuxeries.org) has joined #zumastor 2007-08-12 16:42 -!- erwan_taf(~erwan@konilope.linuxeries.org) has joined #zumastor irc.oftc.net #zumastor log beginning Mon Aug 13 00:00:04 PDT 2007 irc.oftc.net #zumastor log beginning Tue Aug 14 00:00:04 PDT 2007 2007-08-14 06:55 -!- erwan_taf(~erwan@konilope.linuxeries.org) has joined #zumastor 2007-08-14 07:22 -!- erwan_taf(~erwan@konilope.linuxeries.org) has joined #zumastor 2007-08-14 08:40 -!- murb(~murbix@soapstone.yuri.org.uk) has joined #zumastor 2007-08-14 08:40 -!- juuva(juuva@peili.org) has joined #zumastor 2007-08-14 15:55 flipz: ping 2007-08-14 15:57 hi shapor 2007-08-14 15:58 I'll be seeing you at goog in .5 hr or so 2007-08-14 15:58 cant miss the sushi eh? 2007-08-14 15:59 i was going to ask what you thought about breaking all the cli stuff out of ddsnap.c in to ddsnapcli.c 2007-08-14 15:59 true 2007-08-14 15:59 not wild about that at this point 2007-08-14 16:00 first, make the cli lean and mean, get the interface nice, *then* break it into two files 2007-08-14 16:00 breaking it up, then factoring is usually a bad idea 2007-08-14 16:00 i figured its a good start 2007-08-14 16:01 actually, it's a good finish 2007-08-14 16:01 a good start is to get the code right first 2007-08-14 16:02 one day there may be alternative interfaces besides the cli 2007-08-14 16:02 it lets you not look at the popt ugliness when you edit ddsnap ;) 2007-08-14 16:02 make it not ugly 2007-08-14 16:02 make popt not ugly, that is 2007-08-14 16:02 if that means writing a replacement, then fine 2007-08-14 16:02 I have thought a little bit about it 2007-08-14 16:04 well the cli is somewhat complex, it seems to make sense to have it pulled out 2007-08-14 16:04 and then we can change it around 2007-08-14 16:04 something like: argc = parseopts(argc, argv, &argv, &optc, &optv, workbuf); 2007-08-14 16:04 and watch it shrink from 1200+ lines 2007-08-14 16:04 it is easier and better to change it around before pulling out 2007-08-14 16:05 by pulling out you generate a bunch of header cruft that just slows down the refactoring process 2007-08-14 16:05 eh, like 10 lines 2007-08-14 16:05 more 2007-08-14 16:05 anyway, *any* additional layer slows things down 2007-08-14 16:06 you go from zero bogus interfaces, to 1, and increase of 100% 2007-08-14 16:06 how do you like the parseopt arrangement above? 2007-08-14 16:07 workbuf? 2007-08-14 16:07 space to put any new structures non-destructively 2007-08-14 16:07 so arg parsers destroy argv as they work, that sucks 2007-08-14 16:07 um 2007-08-14 16:07 *some* arg parsers destroy argv as they work, that sucks 2007-08-14 16:08 so optc and optv are your configuration ? 2007-08-14 16:08 they are the returned vector of options 2007-08-14 16:09 this function takes as input a vector and count of tokens, returns a vector and count of arg tokens and a vector and count of option tokens/value paris 2007-08-14 16:10 using workbuf for any working space required 2007-08-14 16:10 ah, the size of workbuf needs to be passed 2007-08-14 16:10 and returning the resulting arg count as the result is questionable 2007-08-14 16:10 why not just have it allocate it? 2007-08-14 16:11 because then you need to invent a destructor mechanism 2007-08-14 16:11 cruft that starts piling up 2007-08-14 16:12 libpng does that, it sounds nice as an idea, in practice it is mind numbingly horrible 2007-08-14 16:12 hm 2007-08-14 16:12 that does sound nicer than what we have now 2007-08-14 16:13 anyway, I would say, start by making the libpopt version nicer 2007-08-14 16:13 that is certainly possible 2007-08-14 16:13 because i was just diff'ing my svn 2007-08-14 16:13 noticed that was outstanding 2007-08-14 16:13 i guess i'll just revert it all 2007-08-14 16:14 i still think its nice having it separate 2007-08-14 16:14 if ((optc = parseopts(&argc, &argv, &optv, workbuf, worksize)) == -1) ; 2007-08-14 16:14 better 2007-08-14 16:14 then it bcomes obvious, "this doesnt belong here" 2007-08-14 16:15 but delays the final finish time because the work of making a nice interface between ui and back end slows down 2007-08-14 16:15 so how will that work for error messages? 2007-08-14 16:16 --foo missing required parameter 2007-08-14 16:16 other than standard unix error codes? 2007-08-14 16:17 well we'd want to print that out 2007-08-14 16:17 true, it's a problem if the standard error codes are not adequate 2007-08-14 16:17 if ((optc = parseopts(&argc, &argv, &optv, &error, workbuf, worksize)) == -1) ; 2007-08-14 16:17 heh, it grows 2007-08-14 16:18 it shrunk between first and second iteration though 2007-08-14 16:19 if ((error = parseopts(&argc, &argv, &optc, &optv, workbuf, worksize))) printf(error) and die; 2007-08-14 16:20 the error string could be built up in the workbuf, giving nice dynamic errors 2007-08-14 16:20 not bad 2007-08-14 16:21 would be about a day's work to do this 2007-08-14 16:21 writing the grammar for gnu opts is the hardest part 2007-08-14 16:23 why did you make it &argv 2007-08-14 16:25 hm can options have more than one parameter? 2007-08-14 16:25 --foo 2007-08-14 16:28 because the function returns a new argv that replaces the old one 2007-08-14 16:29 main parameters argv and argc are replaced, optv and optc are local vars 2007-08-14 16:29 other usage is obviously possible 2007-08-14 16:29 yeah 2007-08-14 16:30 one obvious problem is, the scheme for argv and argc does not work if you want to pass say, argv + 1 and argc - 1 2007-08-14 16:30 so needs more thought 2007-08-14 16:42 if ((error = parseopts(argv, argc, &args, &opts, work, worksize))) printf(error) and die; ... where typeof(args) is struct { int count; char *argv[]; } 2007-08-14 16:43 still 6 args but solves the "ignore the first string from argv" problem 2007-08-14 16:44 user can write argc = args->count; argv = args->argv; if they want irc.oftc.net #zumastor log beginning Wed Aug 15 00:00:03 PDT 2007 2007-08-15 10:55 -!- cbsmith(~user@207.47.98.129.static.nextweb.net) has joined #zumastor irc.oftc.net #zumastor log beginning Thu Aug 16 00:00:04 PDT 2007 2007-08-16 20:19 -!- murb(~murbix@soapstone.yuri.org.uk) has joined #zumastor 2007-08-16 20:19 -!- juuva(juuva@peili.org) has joined #zumastor irc.oftc.net #zumastor log beginning Fri Aug 17 00:00:04 PDT 2007 2007-08-17 10:26 -!- crh(ubiqx@c-71-195-35-115.hsd1.mi.comcast.net) has joined #zumastor 2007-08-17 11:12 jiayingz, shapor, there? 2007-08-17 11:20 What handle for Mr. Phillips? 2007-08-17 11:20 flipz 2007-08-17 11:21 That's what I figured... :) 2007-08-17 11:21 Dunno if'n you'll remember me but I did a presentation regarding Samba on GFS at the 2nd Cluster Summit. 2007-08-17 11:21 I do 2007-08-17 11:21 I got a bumper sticker 2007-08-17 11:21 "samba team" 2007-08-17 11:21 welcome 2007-08-17 11:22 Thanks. 2007-08-17 11:22 I've been trying to find time to read the Zumastor whitepaper. 2007-08-17 11:22 it's pretty short :) 2007-08-17 11:22 I'm working for O'Keefe now and that tends to eat time fairly well. 2007-08-17 11:23 He's got me consulting for Cray. 2007-08-17 11:23 so that's what matt is up to 2007-08-17 11:23 did mean to stay in contact, but... 2007-08-17 11:23 Matt, Ken, and to some extent Tilstra. 2007-08-17 11:23 clustering with gfs? 2007-08-17 11:23 Tilstra's been hit by appendicitis. 2007-08-17 11:24 bummer 2007-08-17 11:24 No. Mike and Ken were working on a backup idea and Matt and I were doing storage for Cray. 2007-08-17 11:24 sounds useful 2007-08-17 11:24 well everybody is welcome to use zumastor/ddsnap for backup 2007-08-17 11:24 We were in touch with one company that was doing Samba on GFS but they've been disbanded. 2007-08-17 11:25 I want to look at some of the technology you're developing for an idea I'm working on. 2007-08-17 11:25 I basically need to keep several machines across a WAN in sync (to some extent). 2007-08-17 11:25 by replicating? 2007-08-17 11:26 Essentially, yes. 2007-08-17 11:26 one way or two way? 2007-08-17 11:26 "One way or two"... Yes. ;) 2007-08-17 11:26 The first problem is to get all of the machines in sync without changes being made. 2007-08-17 11:27 reason I asked is, two way replication is a Hard Problem 2007-08-17 11:27 unless you are talking, one volume goes in one direction, another volume goes in the other, that is Easy 2007-08-17 11:27 The next step is to figure out how to keep them in sync once people start changing the files. 2007-08-17 11:27 Yes, I understand it's a difficult problem. 2007-08-17 11:27 I am, to some extent, talking about one volume in one direction. 2007-08-17 11:28 The idea is that each machine would become the master of any given file. 2007-08-17 11:28 The decision being made based on who opened it first. 2007-08-17 11:28 that works very well, except that there is a performance hit on upstream with the current code 2007-08-17 11:28 um 2007-08-17 11:28 wait 2007-08-17 11:28 That's why I'm interested in your code. :) 2007-08-17 11:28 no, you are back to being Hard 2007-08-17 11:29 I'm also talking to Ken (and when he's well) Mike. 2007-08-17 11:29 you are essentially proposiing a Wan-orient distributed filesystem 2007-08-17 11:29 Yep. :) 2007-08-17 11:29 even harder than local, which is already on the extreme side 2007-08-17 11:30 http://www.brocade.com/products/fan/WAFS.jsp 2007-08-17 11:30 They do it by having a single master. 2007-08-17 11:30 ok, well if you are willing to take a run at it, I am willing to offer words of encouragement ;) 2007-08-17 11:30 I would plan on starting there, and then improving the design and code. 2007-08-17 11:30 single master simplifies things considerably 2007-08-17 11:31 Yes, but then there are all of the WAN Acceleration problems. 2007-08-17 11:31 indeed, and all the consistency problems 2007-08-17 11:31 By the way, the SNIA has a working group (the File Area Network [FAN] WG) that is looking at this stuff. 2007-08-17 11:31 I need to get involved with that WG. 2007-08-17 11:31 Any chance you'll be at the Storage Developer's Conference in September? 2007-08-17 11:32 ah, it would be stupid not to go 2007-08-17 11:32 but I have not thought about it yet 2007-08-17 11:32 I'll be there. I'm running the CIFS plugfest this year. 2007-08-17 11:32 sponsored by snia? 2007-08-17 11:33 Yes. http://storage-developer.org/ 2007-08-17 11:34 ah, stallman and rosen will be there 2007-08-17 11:35 I've met Rosen before. 2007-08-17 11:35 jeff "mr ZFS" bonwick 2007-08-17 11:35 Peter Brahm... 2007-08-17 11:35 I've met him by email 2007-08-17 11:35 know peter pretty well 2007-08-17 11:35 looks like old home week 2007-08-17 11:36 Peter proposed something called pCIFS (think pNFS) a few years ago. We may do something with that. 2007-08-17 11:37 eventually we will join the cluster samba hugfest 2007-08-17 11:37 but cluster nfs v3 will come before that 2007-08-17 11:37 at least for our group 2007-08-17 11:37 not very sexy, but useful 2007-08-17 11:38 I've not found CIFS or Samba to be "sexy". Just a difficult job that someone has to do. 2007-08-17 11:38 pretty much describes all of storage 2007-08-17 11:39 Yep. 2007-08-17 11:39 jiayingz doctoral thesis was on wan filesystems 2007-08-17 11:40 what we do now is comparatively low tech, but hard all the same 2007-08-17 11:40 jiayingz is the person you were looking for when you piped up. 2007-08-17 11:40 yes 2007-08-17 11:40 Yeah. It's difficult, but I've done a bit of work with companies like Peribit, Fineground, Certeon... all WAN acceleration companies. 2007-08-17 11:41 I think they have some good ideas, but in the end the problem they wind up solving isn't WAN acceleration, 2007-08-17 11:41 it's extending local NAS over the WAN. 2007-08-17 11:41 That's a good chunk of what they do. 2007-08-17 11:41 a good wan filesystem will tolerate temporary inconsistency... somehow 2007-08-17 11:41 that's where it gets Hard 2007-08-17 11:42 Yes. The WAN Acceleration (WAFS) systems don't, however. They try to make things fast enough over the WAN that 2007-08-17 11:42 they can tollerate the delays caused by forcing consistency. 2007-08-17 11:43 One problem trying to extend CIFS over the WAN is that it is so heavy-weight. 2007-08-17 11:44 yes, much better to start with a simple code base, then merge the ideas once they work 2007-08-17 11:44 A simpler WAN filesystem would be a start. Then add CIFS metadata management on top so that Samba can run. 2007-08-17 11:44 I'd have to build a CTDB clone for the wide area. 2007-08-17 11:44 if I were to tackle it, I would start with ocfs2 2007-08-17 11:44 Hmmm... Why? (curious since I don't know ocfs2 at all) 2007-08-17 11:45 simple code base, it has all the required moving parts, and it does a very good job of cluster cache management, including remastering locks 2007-08-17 11:45 What license is it under? 2007-08-17 11:46 gpl 2007-08-17 11:46 also... it's a shipping, maintained product 2007-08-17 11:46 Nifty. I'll take a look. 2007-08-17 11:46 we will be using it in the follow on project from zumastor, namely zumagrid 2007-08-17 11:47 hi flipx 2007-08-17 11:47 flipz 2007-08-17 11:47 hi jiayingz 2007-08-17 11:47 hi crh 2007-08-17 11:47 Hi 2007-08-17 11:47 jiayingz, meet crh, a samba developer 2007-08-17 11:47 I just saw the topic 2007-08-17 11:47 nice to meet you, crh 2007-08-17 11:48 Distracted... 2007-08-17 11:48 jiayingz, nice to meet you too. 2007-08-17 11:48 Phillip recommended I read your thesis. :) 2007-08-17 11:49 that is too long to read 2007-08-17 11:49 but I have put the code to my web 2007-08-17 11:49 I'm interested. 2007-08-17 11:49 http://www.citi.umich.edu/u/jiaying/NFSv4.r/ 2007-08-17 11:49 it is based on NFSv4 2007-08-17 11:50 Ah... :) That'll be interesting. 2007-08-17 11:50 I haven't worked on CIFS before 2007-08-17 11:50 That's probably a good thing. It has ruined many a good mind. 2007-08-17 11:50 but I always want to work on it, since CIFS and NFS are the most popular two 2007-08-17 11:50 :) 2007-08-17 11:51 It's not a thesis, but it is too long... here's my doco on CIFS: http://ubiqx.org/cifs/ 2007-08-17 11:51 jiayingz is particularly good at dealing with complex, nasty things 2007-08-17 11:51 When I'm at the SNIA conference, any chance we can sync up? 2007-08-17 11:52 hmm, do not know where this reputation comes from. the memory bug you are thinking about? 2007-08-17 11:52 another summer reading project, along with Deathly Hallows 2007-08-17 11:52 I believe I have an autographed one of those 2007-08-17 11:52 that sounds an interesting conf 2007-08-17 11:53 San Jose again 2007-08-17 11:53 we should show up in force 2007-08-17 11:53 4-5 of us 2007-08-17 11:53 Last year there were talks on NFSv4.1, NFSoverRDMA, etc. 2007-08-17 11:53 and we can invite some samba people over to googleplex for lunch at charlie's or something 2007-08-17 11:53 I guess they were given by Netapp people 2007-08-17 11:54 we haven't tested zumastor performance with CIFS 2007-08-17 11:54 brb... 2007-08-17 11:54 maybe that is the time to start that 2007-08-17 11:55 ziayingz, the next thing I am interested in is, what happens if we delete the "bypass" good and run with Peterz's patches 2007-08-17 11:55 according to theory, it should not deadlock on throttle_sem 2007-08-17 11:56 err, bypass code, not bypass good 2007-08-17 11:56 the book looks interesting and a good start for CIFS 2007-08-17 11:56 I need to read mine 2007-08-17 11:57 well, I just got a bookshelf by my desk, I can unpack some books now 2007-08-17 11:57 just bypass in throttle, not the whole throttle? 2007-08-17 11:57 Sorry... 'phone call. 2007-08-17 11:58 jiangz, right 2007-08-17 11:58 Re: Samba: The majority of the Samba Team won't be at this event (some of us will). Instead, they'll be at an event at Google at the end of the month. 2007-08-17 11:58 Re: NetApp. Actually, it was NetApp and Panasas. :) 2007-08-17 11:59 Regarding Samba and Zumastor... You'll want to dig into the CTDB (Cluster TDB) database system. 2007-08-17 11:59 That's how Windows metadata is stored and shared across a Linux cluster by Samba. 2007-08-17 11:59 It's alpha code, but it's there. 2007-08-17 11:59 what if just a single CIFS server with zumastor 2007-08-17 12:00 That should be no problem. 2007-08-17 12:00 crh, which event? 2007-08-17 12:00 (see how closely I follow internal events...) 2007-08-17 12:00 maybe because we are far from mtv 2007-08-17 12:00 There's a Samba/CIFS event at Google starting on the 26th. I won't be there as it's my wife's birthday. 2007-08-17 12:01 crh, I think I even provided the original design description of ctdb 2007-08-17 12:01 Personally, I wish the Team were still doing the SNIA event instead of doing a separate one on their own, but... 2007-08-17 12:01 proposal was to run the tdb on a single server, then export transactions using messages 2007-08-17 12:02 step 2: make it distributed, if anyway ever got that energetic 2007-08-17 12:02 most probably it is still a single server 2007-08-17 12:02 flips, Interesting. To whom? Ronnie, Tridge, Volker, and James Peach I think were the key players in the development. 2007-08-17 12:02 samba mailing list 2007-08-17 12:02 let me google 2007-08-17 12:02 :) 2007-08-17 12:03 http://lists.samba.org/archive/samba-technical/2004-December/038404.html 2007-08-17 12:04 I remember that conversation, now that you point it out. :) 2007-08-17 12:05 that's where it came from all right 2007-08-17 12:05 It probably won't be remembered by the rest of the Team. So it goes... 2007-08-17 12:05 just simple good sense 2007-08-17 12:05 set the bar low and it becomes possible to jump over it :) 2007-08-17 12:05 Yes. 2007-08-17 12:06 That's the important thing for me to remember as I pursue this idea of mine. 2007-08-17 12:06 very much so 2007-08-17 12:06 ocfs2 succeeds where gfs fails just because of that 2007-08-17 12:06 bar is set lower 2007-08-17 12:06 no real fundamental difference 2007-08-17 12:07 in the end, performance came out very impressive 2007-08-17 12:07 interesting that we just talked about that yesterday 2007-08-17 12:07 I'm going to be on vacaction for a week, and I'll be taking some reading material with me. The Zumastor whitepaper will be in the pile. 2007-08-17 12:07 it's very basic stuff, you can cover it in an hour 2007-08-17 12:08 I'll see if I can find some ocfs2 doco to add to the pile. 2007-08-17 12:08 http://oss.oracle.com/projects/ocfs/ 2007-08-17 12:08 Thnax! 2007-08-17 12:09 Thanks for the time as well. I've got some stuff to deal with before I leave in the morning so I've got to disconnect. 2007-08-17 12:09 Hope to keep in touch. :) 2007-08-17 12:10 indeed, see you 2007-08-17 12:10 see you, crh 2007-08-17 14:31 ACTION sets off for the office irc.oftc.net #zumastor log beginning Sat Aug 18 00:00:05 PDT 2007 irc.oftc.net #zumastor log beginning Sun Aug 19 00:00:05 PDT 2007 2007-08-19 01:29 -!- flipz(~daniel@phunq.net) has joined #zumastor 2007-08-19 19:16 -!- flipz(~phillips@phunq.net) has joined #zumastor irc.oftc.net #zumastor log beginning Mon Aug 20 00:00:03 PDT 2007 2007-08-20 04:49 -!- flips(~phillips@207.47.98.129.static.nextweb.net) has joined #zumastor 2007-08-20 05:26 -!- murb(~murbix@soapstone.yuri.org.uk) has joined #zumastor 2007-08-20 05:26 -!- juuva(juuva@peili.org) has joined #zumastor 2007-08-20 06:12 -!- erwan_taf(~erwan@81.80.43.67) has joined #zumastor 2007-08-20 07:30 -!- erwan_taf(~erwan@81.80.43.67) has joined #zumastor 2007-08-20 07:30 op 2007-08-20 07:30 +h 2007-08-20 08:57 -!- erwan_taf(~erwan@81.80.43.67) has joined #zumastor 2007-08-20 10:28 -!- cbsmith(~user@207.47.98.129.static.nextweb.net) has joined #zumastor irc.oftc.net #zumastor log beginning Tue Aug 21 00:00:02 PDT 2007 2007-08-21 00:07 -!- erwan_taf(~erwan@konilope.linuxeries.org) has joined #zumastor 2007-08-21 13:55 -!- erwan_taf(~erwan@konilope.linuxeries.org) has joined #zumastor irc.oftc.net #zumastor log beginning Wed Aug 22 00:00:02 PDT 2007 2007-08-22 04:20 -!- erwan_taf(~erwan@81.80.43.67) has joined #zumastor 2007-08-22 09:14 -!- cbsmith(~user@adsl-75-56-206-36.dsl.lsan03.sbcglobal.net) has joined #zumastor 2007-08-22 09:14 ACTION waves to everyone 2007-08-22 09:19 hi chris 2007-08-22 09:19 No problems announced yet from Dan T? Impressive. 2007-08-22 09:19 ;-) 2007-08-22 11:13 -!- jiayingz(~jiayingz@207.47.98.129.static.nextweb.net) has joined #zumastor 2007-08-22 11:35 -!- erwan_taf(~erwan@konilope.linuxeries.org) has joined #zumastor 2007-08-22 16:20 jiayingz, there? 2007-08-22 16:28 hi flipz 2007-08-22 16:28 hi 2007-08-22 16:29 let me see, it was about... um... 2007-08-22 16:29 which combination of patches works without needing the bypass 2007-08-22 16:30 only the patch to set PF_MEMALLOC for ddsnap-wrk 2007-08-22 16:30 but I haven't checked in yet 2007-08-22 16:30 and then are either of peterz's or christophs patches required? 2007-08-22 16:31 actually neither of them is required 2007-08-22 16:31 after that 2007-08-22 16:31 there is only performance difference 2007-08-22 16:31 theorectical or measured performance difference? 2007-08-22 16:31 without either patch, we will get some page allocation failure in alloc_skb 2007-08-22 16:32 no measured results yet 2007-08-22 16:32 alloc_skb on the write path or the read path? 2007-08-22 16:32 write path 2007-08-22 16:32 when the allocation failure happens, how do we recover? 2007-08-22 16:33 is it order 0 failure, order 1 or what? 2007-08-22 16:33 network layer should cover that, i think 2007-08-22 16:33 order 0, i think 2007-08-22 16:33 the network layer cannot recover if there are enough failures in a row 2007-08-22 16:34 with enough order 0 failures, very bad things will happen 2007-08-22 16:34 so either peterz's or christoph's patch will fix that? 2007-08-22 16:34 i tested the code wo either peter's patch or chritopher's, and haven't seen a problem 2007-08-22 16:35 if order 0 allocations are failing on write, it will eventually cause a block write failure 2007-08-22 16:35 it may be very rare 2007-08-22 16:35 but once a year is already too many 2007-08-22 16:36 why will it cause a block write failure? it is write to socket 2007-08-22 16:36 because one of our server requests will fail 2007-08-22 16:37 because we are not using tcp 2007-08-22 16:37 so which process had the order 0 failure on skb_alloc? 2007-08-22 16:38 order 0 failure on alloc_page? 2007-08-22 16:38 yes 2007-08-22 16:38 ddsnap-wrk 2007-08-22 16:39 right, so there is no ring buffer of sk_bufs, so why does the write not fail entirely? 2007-08-22 16:39 sorry, I think it is order 1 failure 2007-08-22 16:40 yes, that is the only thing that makes sense to me 2007-08-22 16:40 all right, that is not so bad 2007-08-22 16:41 we turned off printouts for order 1 failure 2007-08-22 16:41 interesting that both peterz's and christoph's patches help fix high order fragmentation, I do not think that was intended 2007-08-22 16:41 if we turn that on, we may see those messages 2007-08-22 16:41 so how do we know about the failures? 2007-08-22 16:41 looking at the code 2007-08-22 16:42 the code? 2007-08-22 16:42 you mean the page allocation failures? 2007-08-22 16:42 yes 2007-08-22 16:42 yes, looking at the code in alloc_page 2007-08-22 16:43 how does that tell us that failures occured? 2007-08-22 16:43 we saw that problem before without setting PF_MEMALLOC for ddsnap-wrk 2007-08-22 16:43 after setting it, it goes to no_page 2007-08-22 16:44 no_page? 2007-08-22 16:44 oh 2007-08-22 16:44 the label 2007-08-22 16:44 yes 2007-08-22 16:44 let me look at that 2007-08-22 16:44 I can turn on the printouts to verify 2007-08-22 16:44 that would be interesting 2007-08-22 16:45 wow, I do not like the way nopage returns page instead of returning NULL 2007-08-22 16:46 page should be null in that case 2007-08-22 16:46 I know, 2007-08-22 16:46 I don't like that coding style unless there is a good reason 2007-08-22 16:46 which there is not here 2007-08-22 16:46 :) 2007-08-22 16:47 I think __GFP_NOWARN is controlled by a config option somewhere 2007-08-22 16:48 is it passed to alloc_page? 2007-08-22 16:48 anyway, so you are sure that we never see order 0 failures on write (with PF_MEMALLOC on wrkr), whether we have peterz's, christoph's, or no vm fixups applied? 2007-08-22 16:49 something like that 2007-08-22 16:49 no 2007-08-22 16:49 I haven't seen any printks 2007-08-22 16:49 :) 2007-08-22 16:49 good answer 2007-08-22 16:50 so for the next week or so maybe we should test without either perterz's or christoph's patches 2007-08-22 16:50 try to get an idea what breaks 2007-08-22 16:50 I will keep my test running 2007-08-22 16:51 and if we remove all our throttling, then lots of things break, right? 2007-08-22 16:51 I looked at the patches in our repository 2007-08-22 16:51 there is some code in ddsnap.vm.fix we may want to move to ddsnap-2.6.21 2007-08-22 16:51 that is the code to set PF_MEMALLOC for ddsnap server 2007-08-22 16:52 yes 2007-08-22 16:52 removing throttle... I haven't tried yet. I only tried removing bypass 2007-08-22 16:52 that is not a fix, it is part of the ddsnap patch 2007-08-22 16:52 yes 2007-08-22 16:53 then all things left in ddsnap.vm.fix is peter's patches 2007-08-22 16:53 ok, the prediction is, if we remove the throttle entirely, bad things happen 2007-08-22 16:53 we should test that 2007-08-22 16:53 we are not sure yet 2007-08-22 16:53 let's test that right now 2007-08-22 16:53 it should not take long to run into trouble 2007-08-22 16:53 we did not have throttle before, and things were OK 2007-08-22 16:53 ok 2007-08-22 16:56 we also want to work on a test case that shows when peter's patches solve the problem 2007-08-22 16:56 the receiver case 2007-08-22 16:56 yes. That will be a lot of work 2007-08-22 16:56 we have a few easier things to test first 2007-08-22 16:56 meanwhile I will work some more on the generic throttling patch, which I still think we need 2007-08-22 16:57 we need proof of that of course 2007-08-22 16:57 agree 2007-08-22 17:44 flipz? 2007-08-22 18:32 hi jiayingz 2007-08-22 19:14 So, what did I miss? 2007-08-22 19:21 lots of chat about throttling etc 2007-08-22 19:24 flipz: I noticed a fair bit of throttling stuff. irc.oftc.net #zumastor log beginning Thu Aug 23 00:00:02 PDT 2007 2007-08-23 01:04 -!- erwan_taf(~erwan@81.80.43.67) has joined #zumastor 2007-08-23 04:57 -!- erwan_taf(~erwan@81.80.43.67) has joined #zumastor 2007-08-23 12:50 -!- erwan_taf(~erwan@konilope.linuxeries.org) has joined #zumastor 2007-08-23 14:48 -!- erwan_taf(~erwan@konilope.linuxeries.org) has joined #zumastor 2007-08-23 19:16 -!- bob(~bob@207-237-10-120.c3-0.nyw-ubr3.nyr-nyw.ny.cable.rcn.com) has joined #zumastor 2007-08-23 19:17 hi. i have some very naive questions about what was (I suppose) ddsnap at some point 2007-08-23 19:18 hello? irc.oftc.net #zumastor log beginning Fri Aug 24 00:00:02 PDT 2007 2007-08-24 01:11 -!- erwan_taf(~erwan@81.80.43.67) has joined #zumastor 2007-08-24 08:30 -!- erwan_taf(~erwan@81.80.43.67) has joined #zumastor 2007-08-24 16:25 -!- erwan_taf(~erwan@konilope.linuxeries.org) has joined #zumastor irc.oftc.net #zumastor log beginning Sat Aug 25 00:00:02 PDT 2007 2007-08-25 04:18 -!- erwan_taf(~erwan@konilope.linuxeries.org) has joined #zumastor 2007-08-25 12:57 -!- daniel__(~phlipz@phunq.net) has joined #zumastor 2007-08-25 16:03 -!- erwan_taf(~erwan@konilope.linuxeries.org) has joined #zumastor irc.oftc.net #zumastor log beginning Sun Aug 26 00:00:02 PDT 2007 2007-08-26 05:14 -!- phlyps(~phlipz@phunq.net) has joined #zumastor 2007-08-26 10:19 hello? 2007-08-26 13:30 hello? 2007-08-26 13:39 oh...i'm gues1771 2007-08-26 13:39 who would hae guessed 2007-08-26 13:39 either way none of you are here, because no-one is ever here 2007-08-26 13:48 -!- juuva_(juuva@peili.org) has joined #zumastor 2007-08-26 13:53 -!- Guest1771(~bob@207-237-10-120.c3-0.nyw-ubr3.nyr-nyw.ny.cable.rcn.com) has left #zumastor irc.oftc.net #zumastor log beginning Mon Aug 27 00:00:02 PDT 2007 2007-08-27 01:21 -!- erwan_taf(~erwan@81.80.43.67) has joined #zumastor 2007-08-27 05:09 -!- erwan_taf(~erwan@81.80.43.67) has joined #zumastor 2007-08-27 12:44 -!- erwan_taf(~erwan@konilope.linuxeries.org) has joined #zumastor 2007-08-27 15:06 -!- erwan_taf(~erwan@konilope.linuxeries.org) has joined #zumastor irc.oftc.net #zumastor log beginning Tue Aug 28 00:00:02 PDT 2007 2007-08-28 01:12 -!- erwan_taf(~erwan@81.80.43.67) has joined #zumastor 2007-08-28 01:32 -!- erwan_taf(~erwan@81.80.43.67) has joined #zumastor 2007-08-28 08:41 -!- murb(~murbix@soapstone.yuri.org.uk) has joined #zumastor 2007-08-28 10:36 -!- erwan_taf(~erwan@konilope.linuxeries.org) has joined #zumastor 2007-08-28 13:15 shapor, ping? 2007-08-28 14:02 -!- jiayingz(~jiayingz@207.47.98.129.static.nextweb.net) has joined #zumastor 2007-08-28 14:09 hi jiayingz 2007-08-28 14:11 hi phlyps 2007-08-28 14:11 hi 2007-08-28 14:12 tried any new tests this week? 2007-08-28 14:12 I have been running that test on shapor's machine 2007-08-28 14:12 but did not get any deadlock 2007-08-28 14:13 clameter patch without throttling? 2007-08-28 14:13 I once got some deadlock state, but then the test program that I used to alloc memory got killed by the kerenl and after that, everything went ok 2007-08-28 14:13 an oom 2007-08-28 14:14 lameter's patches with throttling 2007-08-28 14:14 but without bypass 2007-08-28 14:14 that's good info 2007-08-28 14:14 now we need to try clameter + no throttling at all 2007-08-28 14:14 then i tried to setup serial console on my testing machine several times 2007-08-28 14:14 but no luck so far 2007-08-28 14:15 if you can get the traceback on the screen, then a digital photo will do fine 2007-08-28 14:15 that would be a little wierd 2007-08-28 14:15 we can copy it down by hand though 2007-08-28 14:16 and it will not be weird 2007-08-28 14:16 shapor's machine has hda and 2 giga mem 2007-08-28 14:16 don't know if that is the reason 2007-08-28 14:16 hard to say 2007-08-28 14:16 I have trace from uml 2007-08-28 14:16 it is very easy for deadlocks like this to vanish temporarily 2007-08-28 14:16 that shows something 2007-08-28 14:17 want to paste it here? 2007-08-28 14:17 here? 2007-08-28 14:17 sure 2007-08-28 14:17 can i send a file on irc? 2007-08-28 14:17 just paste it into the channel 2007-08-28 14:17 it's not very long 2007-08-28 14:17 the part that shows the deadlock traceback 2007-08-28 14:18 also, I posted this: http://lkml.org/lkml/2007/8/28/324 2007-08-28 14:18 the throttling patch 2007-08-28 14:18 not quite perfect, and not tested 2007-08-28 14:19 but Evgeniy is having troulble with with concepts of atomic operations, so I posted it 2007-08-28 14:19 note: my own atomic operation has a big whole in it 2007-08-28 14:19 ok 2007-08-28 14:19 let's see who notices first 2007-08-28 14:19 big hole that is 2007-08-28 14:21 ddsnap-wrkr - S 08048514 0 1222 1 (L-TLB) 2007-08-28 14:21 0f9c5d7c 00000001 00000000 0f613c9c 0805e6eb 083d37bc 0f9c5d7c 0f610000 2007-08-28 14:21 0f610000 0f613cc0 0805b2b9 083d32c0 0f9c5880 0f610000 0f610000 08b625e0 2007-08-28 14:21 0f9c5880 083d32c0 0f613d08 082a9d13 083d32c0 0f9c5880 083d32c0 0f613cfc Call Trace: 2007-08-28 14:21 0f613c88: [<0805e6eb>] switch_to_skas+0x3b/0x90 2007-08-28 14:21 0f613ca0: [<0805b2b9>] _switch_to+0x49/0xa0 2007-08-28 14:21 0f613cc4: [<082a9d13>] schedule+0x243/0x520 2007-08-28 14:21 0f613d0c: [<082aa7c5>] schedule_timeout+0x75/0xc0 2007-08-28 14:21 0f613d40: [<081ec8b3>] sock_wait_for_wmem+0xc3/0xe0 2007-08-28 14:21 0f613d6c: [<081ec956>] sock_alloc_send_pskb+0x86/0x1e0 2007-08-28 14:21 0f613d9c: [<081ecadc>] sock_alloc_send_skb+0x2c/0x30 2007-08-28 14:21 0f613db8: [<082632b2>] unix_stream_sendmsg+0x1c2/0x350 2007-08-28 14:21 0f613e0c: [<081e93d7>] do_sock_write+0x97/0xd0 2007-08-28 14:21 0f613e30: [<081e9483>] sock_aio_write+0x73/0x80 2007-08-28 14:21 0f613e94: [<081e5f03>] rwpipe+0x133/0x160 2007-08-28 14:21 0f613f50: [<081e7290>] worker+0x230/0x450 2007-08-28 14:21 0f613fb4: [<0806ef5e>] run_kernel_thread+0x2e/0x50 2007-08-28 14:21 0f613fe0: [<0805e7a0>] new_thread_handler+0x60/0x90 2007-08-28 14:21 0f613ffc: [<00000000>] _sinittext+0xf7fb7000/0x20 2007-08-28 14:22 that is the key part 2007-08-28 14:22 let's see what sock_wait_for_wmem does 2007-08-28 14:23 just calls schedule_timeout, basically 2007-08-28 14:24 and what wakes it up again? 2007-08-28 14:25 that can look different from a real machine, but the general path is similar 2007-08-28 14:25 -!- cbsmith(~user@207.47.98.129.static.nextweb.net) has joined #zumastor 2007-08-28 14:25 sock_wait_for_wmem is net core 2007-08-28 14:25 SOCK_NOSPACE 2007-08-28 14:26 someting interesting is on shapor's machine, I did not even see any PF_LESS_THROTTLE process enters congestion_wait 2007-08-28 14:26 I saw a lot of that message on my testing machine 2007-08-28 14:27 it sounded like an ordinary oom 2007-08-28 14:27 did you run the cyclic_mmap program? 2007-08-28 14:28 you mean cyclic_anon? 2007-08-28 14:28 right 2007-08-28 14:28 yes 2007-08-28 14:28 with too high a value I think 2007-08-28 14:28 causing oom 2007-08-28 14:28 if that is oom, why no process got killed? 2007-08-28 14:29 the cyclic_anon program exited? 2007-08-28 14:29 I might be wrong about the oom 2007-08-28 14:29 oh yes. I did not see cyclic_anon on sysrq-t 2007-08-28 14:30 ok, that explains it 2007-08-28 14:30 oom_killer waits a _long_ time before actually killing anything 2007-08-28 14:30 but the system did not come back after that 2007-08-28 14:30 it's pretty useless 2007-08-28 14:30 oh, ok, that sounds like a deadlock 2007-08-28 14:30 so do we have a trace? 2007-08-28 14:31 sysrq-t trace? yes, but that is on uml 2007-08-28 14:31 i got that from mconsole 2007-08-28 14:31 no trace on shapor's machine? 2007-08-28 14:31 no 2007-08-28 14:31 oh yes, some trace 2007-08-28 14:32 but the system went back after killing cyclic_anon 2007-08-28 14:32 well, a jpg photo of a backtrace on your real test machine would make me happy 2007-08-28 14:33 the problem is we do not always see ddsnap trace on real machine 2007-08-28 14:34 i mean my testing machine 2007-08-28 14:34 it may take a while 2007-08-28 14:34 that is ok 2007-08-28 14:34 do we have any workarround? 2007-08-28 14:34 then we need a nice high res digital camera and take pictures 2007-08-28 14:35 workaround for? 2007-08-28 14:35 maybe we can have drake try it on gsd2950 2007-08-28 14:35 if we have a machine on which we know the deadlock occurs, we should use that 2007-08-28 14:36 it is interesting to try to find it on new machines, but that can take an unknown amount of time 2007-08-28 14:36 agree. but we can do it in parallel 2007-08-28 14:37 there are several things we can do in parallel 2007-08-28 14:37 this is low quality code -> http://lxr.linux.no/source/net/core/sock.c#L1100 2007-08-28 14:37 like much of the net layer 2007-08-28 14:37 continue to try it on my machine 2007-08-28 14:37 yes, that is easiest 2007-08-28 14:37 easy things are nice 2007-08-28 14:37 turn off throttle totally and try it again on shapor's machine 2007-08-28 14:38 yes 2007-08-28 14:38 and try it on drake's machine 2007-08-28 14:38 sure 2007-08-28 14:38 :) 2007-08-28 14:38 it is nice to have machines working hard instead of people 2007-08-28 14:39 ok, actually I still keep bypass for throttle on shapor's machine 2007-08-28 14:39 because otherwise, the trace would look different 2007-08-28 14:39 this is the part of wait_for_wmen that does the work: if (atomic_read(&sk->sk_wmem_alloc) < sk->sk_sndbuf) 2007-08-28 14:39 yes 2007-08-28 14:41 jiayingz, can you describe for me the conditions that caused the traceback you pasted above? 2007-08-28 14:41 which patches did we have? 2007-08-28 14:42 I think that was with lameter's patch, but without throttle, on uml 2007-08-28 14:45 just making sure 2007-08-28 14:45 so I will post it to christoph and see what he says 2007-08-28 14:45 it is not hard to reproduce the problem on uml 2007-08-28 14:46 I can have a trace with throttle 2007-08-28 14:46 just to make sure 2007-08-28 15:09 jiayingz, still there? 2007-08-28 15:09 yes 2007-08-28 15:10 there is more to the story of the traceback above 2007-08-28 15:10 somewhere, there are other kernel tasks trying to write out memory 2007-08-28 15:10 or trying to scan for clean pages 2007-08-28 15:10 I think that is ddsnap agent 2007-08-28 15:11 quite likely 2007-08-28 15:11 ddsnap D 08048514 0 4635 4634 (NOTLB) 2007-08-28 15:11 0f9c51bc 00000001 00000000 08aafb50 0805e6eb 0f9c4bdc 0f9c51bc 08aac000 2007-08-28 15:11 08aac000 08aafb74 0805b2b9 0f9c46e0 0f9c4cc0 08aac000 08aac000 0b073a00 2007-08-28 15:11 0f9c4cc0 0f9c46e0 08aafbbc 082a9d13 0f9c46e0 0f9c4cc0 0f9c46e0 08aafba0 Call Trace: 2007-08-28 15:11 08aafb3c: [<0805e6eb>] switch_to_skas+0x3b/0x90 2007-08-28 15:11 08aafb54: [<0805b2b9>] _switch_to+0x49/0xa0 2007-08-28 15:11 08aafb78: [<082a9d13>] schedule+0x243/0x520 2007-08-28 15:11 08aafbc0: [<082aa79e>] schedule_timeout+0x4e/0xc0 2007-08-28 15:11 08aafbf4: [<082aa721>] io_schedule_timeout+0x11/0x20 2007-08-28 15:11 08aafbfc: [<080a86a1>] congestion_wait+0x81/0xe0 2007-08-28 15:11 08aafc34: [<080a39ce>] balance_dirty_pages+0xae/0x160 2007-08-28 15:11 08aafc8c: [<080a3b20>] balance_dirty_pages_ratelimited_nr+0x40/0x50 2007-08-28 15:11 08aafc98: [<080a01ce>] generic_file_buffered_write+0x29e/0x700 2007-08-28 15:11 08aafd44: [<080a08c2>] __generic_file_aio_write_nolock+0x292/0x5a0 2007-08-28 15:11 08aafdd4: [<080a0c24>] generic_file_aio_write_nolock+0x54/0xc0 2007-08-28 15:11 08aafe0c: [<080bcf2c>] do_sync_write+0xdc/0x130 2007-08-28 15:11 08aafec4: [<080bd050>] vfs_write+0xd0/0x190 2007-08-28 15:11 08aafef4: [<080bd2e2>] sys_pwrite64+0x72/0x80 2007-08-28 15:11 08aaff18: [<0805ebe2>] handle_syscall+0x92/0xc0 2007-08-28 15:11 08aaff6c: [<0807321a>] handle_trap+0x2a/0x140 2007-08-28 15:11 08aaff94: [<080739a5>] userspace+0x1e5/0x240 2007-08-28 15:11 08aaffe4: [<0805e854>] fork_handler+0x74/0x90 2007-08-28 15:11 08aafffc: [<830c5d8b>] 0x830c5d8b 2007-08-28 15:11 oh, it got hung up on congestion_wait, yes that is the rest of the story 2007-08-28 15:11 and that is with or without PF_LESS_THROTTLE? 2007-08-28 15:12 that is the story without throttle, so it is different from what we saw 2007-08-28 15:12 with PF_LESS_THROTTLE for ddsnap server and ddsnap-wrk 2007-08-28 15:12 ddsnap agent does not run with PF_LESS_THROTTLE 2007-08-28 15:13 the log is in my home dir /home/jiayingz/sysrq-nothrottle 2007-08-28 15:13 it does not really need to 2007-08-28 15:14 the agent that is 2007-08-28 15:14 right irc.oftc.net #zumastor log beginning Wed Aug 29 00:00:01 PDT 2007 2007-08-29 00:51 -!- erwan_taf(~erwan@81.80.43.67) has joined #zumastor 2007-08-29 10:05 is http://techreport.com/articles.x/9312/1 like the sram described in the paper? 2007-08-29 17:03 -!- cbsmith(~user@207.47.98.129.static.nextweb.net) has joined #zumastor irc.oftc.net #zumastor log beginning Thu Aug 30 00:00:02 PDT 2007 2007-08-30 01:22 -!- erwan_taf(~erwan@81.80.43.67) has joined #zumastor 2007-08-30 14:11 -!- cbsmith(~user@207.47.98.129.static.nextweb.net) has joined #zumastor 2007-08-30 17:58 -!- fmayhar(~fmayhar@207.47.98.129.static.nextweb.net) has left #zumastor irc.oftc.net #zumastor log beginning Fri Aug 31 00:00:02 PDT 2007 2007-08-31 01:11 -!- erwan_taf(~erwan@81.80.43.67) has joined #zumastor 2007-08-31 08:48 -!- fmayhar(~fmayhar@207.47.98.129.static.nextweb.net) has joined #zumastor 2007-08-31 11:05 -!- erwan_taf(~erwan@konilope.linuxeries.org) has joined #zumastor