OK, that makes sense.
I think you may need to use DFSee to put the LVM information on the stick and maybe even assign the drive letters, that way when you eject one partition the others are automatically ejected.
As a test I just put a 250 GB hard drive into a USB caddy, used DFSee to make three partitions assign drive letters and LVM information. I then ejected the unit and plugged it in again, the three partitions were recognised with the assigned drive letters. Now comes the only difference, only one is a 2 GB partition with the others being 123 GB each (there is a reason for that, this drive will be a copy of one of our workstations). After formatting the partitions I am able to highlight any one of them in the drives object, click on eject and all three are ejected without problems.