God bless LVM

Correct me if I’m wrong (and I really want to be wrong).
Is that whole LVM crap expects me to be forecaster/fortune-teller?

I wanted to upgrade some squeeze based virtual machine to wheezy. Sounds nice, yeah?
So before the upgrade you should ensure that you have backup, right?
Assuming that whole system is LVM based all you have to do before upgrade is LVM snapshot.
I did it and I was quite surprised that I had to declare its size.
I did it basing on assumption that upgrade shouldn’t take more than let say 5GB.

Tada!

mastersrv:~# lvs
  LV       VG      Attr   LSize   Origin  Snap%  Move Log Copy%  Convert
  backup   virtual -wi-ao  50,00g                                       
  mainsrv  virtual -wi-ao 300,00g                                       
  redminbu virtual swi-a-   5,00g redmine   0,00                        
  redmine  virtual owi-a-  19,53g

Now I did upgrade redmine virtual machine and this is what I got:

mastersrv:~# lvs
  LV       VG      Attr   LSize   Origin  Snap%  Move Log Copy%  Convert
  backup   virtual -wi-ao  50,00g                                       
  mainsrv  virtual -wi-ao 300,00g                                       
  redminbu virtual Swi-I-   5,00g redmine 100.00                        
  redmine  virtual owi-ao  19,53g

These attr flags mean “Snapshot invalid” and “Invalid snapshot”.
End of the story… this is last time I actually tried that method of backup.
Happily I have also some traditional backup (mysqldump+rsync) but I’m really surprised that LVM works this way.
There was plenty of free space to enlarge that volume and this is what you get in your logs:

[13402279.777238] device-mapper: snapshots: Invalidating snapshot: Unable to allocate exception.

Unable to allocate exception? Did it really try to do it?

mastersrv:~# pvs
  PV         VG      Fmt  Attr PSize   PFree  
  /dev/md0   virtual lvm2 a-   884,01g 514,46g

I don’t think so.

  • Bookmark and Share

14 thoughts on “God bless LVM

  1. grandpa

    well, no, that’s not the way to do it…

    the snapshot serves as a way to freeze the FS so the backup will be consistent. so you take a snapshot and use something like tar to backup the snapshot data

    Reply
    1. fEnIo Post author

      I still think that making snapshot totally unusable because of many changes on original filesystem simply sucks. Especially since there is a plenty of free space to simply grow that volume.

      Reply
        1. XANi

          Um, no, if you change 10 gigs of data, you need to have that 10 gigs of space free and if you have 500 gigs of space free just set snapshot = original volume size.

          If you would make 20 G btrfs partition, put 15 G in it, snapshot and then change 10 G of data in it, same thing would happen.

          Altho autoresize of partition/fs would be nice, it’s easy to shoot yourself in the foot with that, and its also pretty easy to implement that in userspace

          Reply
  2. Jon Severinsson

    Well, when you made the snapshot, you did tell it to under no circumstances grow larger than 5 GB, and now you complain that it did’t automatically grow any larger. But ofcourse it didn’t, you told it not to.

    If you want it to grow as large as it need too, just tell it that it may (by specifying a maximum snapshot size equal to the size of the volume you are snapshoting). That way the snapshot will not go invalid.

    This is not a case of LVM not knowing how to do the right thing, it is a case of you explicitly telling it not to, and then complaining about it…

    Reply
    1. fEnIo Post author

      As I said I was surprised that it asks me for the size of snapshot. I thinks it’s simply stupid and it should autogrow the volume whenever there is free space not assigned to any legitimate data.

      Reply
  3. Franklin Piat

    Are you actually blaming LVM for your mistake ? Let me explain:
    What prevented you from allocating way more space that just “roughtly expected” ? (why not 100% if you don’t want to estimate the space needed ?)
    Did you consider that you are touching lots of blocks ? (download, uncompress …) if you touch an application or a database are sure you are not going to reorganise the whole data/database ?

    Yes, LVM is doing the sensible thing: Guaranty disk allocation *before* your manipulation.
    So, if you are in a large organisation and many people use snapshot, each of them can reserve some space for each snapshot.
    In the worse case, one try to *create* a snapshot but that fails because he can’t reserve that amount of space.

    summary: don’t assume: Plan then Test, then do in your production environnment !

    Reply
    1. fEnIo Post author

      Come on… this is *logical* volume manager and I would expect it to manage my volumes logically. As I said question about size of the snapshot was *strange* for me and I suppose that most users answer “whatever dude… it’s snapshot”.
      Then you silently lose that data cause there are too many changes in original filesystem.
      I just think it’s not *logical* nor user friendly.
      Sure I should read documentation before deciding on the size of the volume, but I just thought that it’s logical. Just change the name to sophisticated volume manager and I’m going to read its whole documentation before taking any step.

      Reply
      1. Arno

        Behind-the-scenes “intelligent” automation is never simple and rarely logical (if ever). LVM stands for (logical volume) manager, not logical (volume manager).

        I can understand that most users would answer “whatever dude”, but Debian (as Unix in general) expects a sysadmin to have more skills and consideration than a typical user.

        Barring the automated resizing, would you rather have had LVM return ENOSPC when the snapshot ran out of room?

        Reply
        1. fEnIo Post author

          Fine. I’ve got more skills than typical user (I had backup made the other way) but only when it comes to sysadmin skills. When it comes to understanding naming convention I’m a typical user.

          In fact “intelligent” automation in my case would mean to simply enlarge snapshot as long as there is enough room for it and it won’t hurt legitimate data. I still can’t believe I was asked for its size.

          Reply
      2. rozie

        I don’t understand. You told the program to make a snapshot of some data on much too small space. It tried, and failed. It reported in log that failed, it indicated the problem. Snapshots were marked as invalid. What’s the problem?

        Yeah, it could display on the screen big, blinking message that snapshot failed. It also could recommend size for the space needed to snapshot given volume. That’s true.

        But expecting that it will automagically allocate more space? C’mon, it’s like expecting that if you run ouf of space on one partition, and having “unpartitioned” disk space, system would automagically partition those “free” space make FS on it and mount it. Even Windows does not such a thing. But it does (at least used to) similar thing: it always installed own bootloader, because “there was no bootloader”. Great feature, isn’t it?

        Reply
        1. fEnIo Post author

          That’s right. You don’t understand.
          I made a snapshot and it was perfectly fine until I made so many changes on the original filesystem that the snapshot size was to small to keep them.
          Then it silently marked it as invalid.

          And yes… I would expect it to use unallocated space cause I really think that it should be more intelligent solution than simple partitions.

          Reply

Leave a Reply

Your email address will not be published. Required fields are marked *


− 4 = three

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>