refracta2usb / snapshot2usb version: 2.4.x CONTENTS Intro Quick Start Important Notes 1. UEFI support 2. Uh-oh, I previously used dd or cat (or other) to directly image the usb drive with an isohybrid file. I followed all the steps, but it won't boot. 3. Network security for static IP or wireless password 4. Multiboot-usb toram Update the live image on a previously prepared drive. Explanation of persistence Basic tasks Format the usb drive. Create a live usb from live-CD iso file. Bootloader information. Persistent Partition Persistent Loopback File Encrypted Persistence Tools Edit boot menu (Re)install bootloader Patch initrd for rw-media. Mount/unmount loopfiles Delete stray loop devices. Delete live systems from the usb. Examples Sample directory structures for various setups on the usb drive. Sample persistence.conf Sample boot menu entries ____________________ ___ INTRODUCTION ___ refracta2usb allows you to create a live-USB system on a thumb drive using a live-CD iso file or directly from a running live system. It also allows you to replace the system on an existing live-USB drive with a system from another iso file, so you can easily update your live system with your latest snapshot. There are advanced options for creating and using persistent partitions or files, with or without encryption, read-write media, and probably more by the time you read this. Think of this program as a Swiss army knife - it's a collection of tools, some of which you may need, some you may never use, and some whose uses are mutually exclusive. The main menu is not a sequential list of tasks, although it may appear that way. *********************************************** You must be root! To start refracta2usb-2.0+, run from a root terminal (or with sudo) refracta2usb For debugging problems, run refracta2usb -d ************************************************************** ___________________ ___ QUICK START ___ Plug in usb thumb drive; do not open or mount it. Start refracta2usb and select the device you want to use. FOR A NEW THUMB DRIVE At the Main Menu: Format the stick (if needed): First partition, fat32, boot flag on. Label is optional. Second partition is optional. For standard debian-live persistence, a label is necessary. The label can be "persistence" or other name that you choose. Format should be ext2 (or ext3,4). Create Live-USB: Either from ISO image or from the running Live session. This will copy the live system to a named folder on the first partition. Persistent (Optional) Choose this if you want to use a separate partition for persistence. (Allows you to save files that will still be there after a reboot.) Loopback (Optional) Choose this if you want to use a loopback file as a persistent volume. This allows you to have different persistent volumes for different live systems or or different volumes for the same live system, but with different configurations. Bootloader (No longer in the Main menu) If you're creating a new live-usb, the bootloader will be added automatically. If you're adding another live system to an existing live-usb that already works, it is assumed that you have a good working bootloader, and only the boot menu is changed to add an entry for the added live system. Done. ______________________ UEFI SUPPORT If you create a live-usb from a uefi-compatible iso file, the /boot and /efi directories will be copied to the usb drive, and the grub boot menu will open in a text editor along with the syslinux boot menu. Additional live systems added to a multi-boot usb will not overwrite the working efi and boot files by default. If you do allow the files to be replaced, your old grub menu will open in a window for you to copy/paste any custom grub menu entries that you want to keep. Added systems only need to be listed in the grub boot menu to boot on uefi hardware - they do not need their own efi files. IMPORTANT NOTE ABOUT USING ISOHYBRID IMAGES If Your Thumb Drive Was Previously Imaged With Isohybrid Image using dd, cat or rawrite command: Zero the first 2MB of the drive before running refracta2usb dd if=/dev/zero of=/dev/sdX bs=512 count=4096 where sdX is replaced with the correct device name for your usb. YOU WILL LOSE ALL DATA ON THE SELECTED DEVICE. TAKE CARE TO ZERO THE CORRECT DRIVE !!! Run refracta2usb. Format the drive. You'll need to create a new partition table before you can make partitions. Follow directions for a New thumb drive. (see #1) You may need to repeat this procedure, and you may need to zero more than 4096 blocks. You also may need to reinstall the syslinux bootloader, and just in case that doesn't work, you could install the alternate bootloader. IMPORTANT NOTE ABOUT NETWORK SECURITY If you made a snapshot of a system that has static network settings or wireless password information, and you used a version of refractasnapshot before 9.2.0, your network information was copied into your snapshot. When you boot that snapshot, the saved network files will be used for boot and accessible from the system only if you boot with the option, ip=frommedia. The presence of such network settings in the snapshot could be a security risk, if you distribute that snapshot to other people. Versions of refractasnapshot from 9.2.0 do not copy those settings for wicd and /etc/network/interfaces unless you edit the config file to specifically allow it. Versions from 10.0.3 also exclude settings for Gnome NetworkManager, and from 10.2.12 exclude Connman settings. Other network managers may store the information in places that do get copied. If you do allow it, and you want to use your custom network settings in your live-usb, the boot option can be added by checking the appropriate box when it's presented to you. MULTIBOOT USB TORAM Using the toram option with a multiboot live-usb causes the entire first partition to be copied to RAM. This fails if the partition is larger than the available RAM. Note that this does NOT occur if you use an intact iso file with the findiso option (ISO_2 in main menu) For a live system made from unpacked iso or from live session, add the following to the boot command: toram=filesystem.squashfs For a live system made from intact iso and using findiso option, just adding 'toram' will work (ISO_2). ______________________ TO UPDATE A PREPARED DRIVE WITH A NEWER ISO IMAGE Multiboot configuration is assumed. All live systems added to the flash drive are put in their own named directory. All the files for the bootloader are kept separate from this. To replace an existing live system with a new one, just delete the folder that contains that system and add the new system using one of the "Create" options in the main menu. For convenience, you can delete folders from within the program - select "Open device in a root file manager." from the Tools/Utilities menu. Done. Warning: If you update a system that uses full persistence, you may need to delete some or all of the system files on the persistent filesystem, depending on how different the updated iso is from the one it is replacing. ________________________________ ___ EXPLANATION OF PERSISTENCE ___ A live-CD or live-USB normally runs with a read-only filesystem. In order to have an area with read-write permissions for saving file, a feature called persistence can be enabled. There are two basic kinds of persistence - persistent /home, which allows a user to save files and desktop configurations in their own home directory, and full persistence, which allows saving system files in addition to home files. The choice of full or home persistence is determined by the contents of the config file, persistence.conf, which gets placed in the root of the persistent filesystem. That filesystem may be contained in a separate partition from the system, or it may be in a loopback file. In either case, any saved files are kept on the persistent filesystem, and the system knows to use the files it finds there. Note that with full persistence, the read-only filesystem is not changed, but the system knows to use any files it finds on the persistent filesystem, such as altered config files or added programs. Refracta2USB provides several different ways of setting up persistence. The simplest case is to have a second partition for saving files. This can be set up from the Main Menu in Refracta2USB. Other methods are available in the Advanced Menu. Encryption can be used on persistent partitions or loopback files. See below. _______________________ _____ BASIC TASKS _____ New thumb drive: Plug in usb thumb drive; do not open or mount it. Start refracta2usb and select the device you want to use. (If you start the program before you plug in the drive, you can Rescan.) At the Main Menu: 1. Format the stick. First partition must be fat32 (vfat) with boot flag on. If the usb stick was formatted as fat32 to begin with, you can just mark it bootable and optionally give it a label (recommended) If you want a partition that's writable for saving files, you should resize the fat32 partition to make room for the second partition. The recommended filesystem type for the second partition is ext2. For standard debian-live persistence, the label should be "persistence", however this program will automatically use any valid label you give the filesystem. (See #4 below.) 2. Create Live-USB: There are four ways to do this. In each case, files for the live system are put in a named directory in the root of the first partition. ISO_1: Copy the contents of the iso file. ISO_2: Copy the intact iso file and boot it with the 'findiso' option. This is useful if you want to have a copy of the iso file readily available for burning. Live_1: Copy from a running live-usb Live_2: Copy from a running live-cd Note: When copying from a running live system, any changes you make to the running system in that session will NOT be copied, even if you're running a persistent session. The source of the copy is the mounted (read-only) image of the live system. Bootloader If you're starting with a new flash drive, the program will see that there is no syslinux folder in the root of the device. A skeleton syslinux folder will be copied to the flash drive with the menu config and help files. Syslinux modules will be copied from the host system, the syslinux command will be issued, and boot code will be copied to the master boot record of the flash drive. Doing it this way eliminates the problem of mismatched modules and boot code from different versions of syslinux. Consequently, this should work on jessie/sid and wheezy. Note for anyone using a different bootloader: If you don't want your bootloader touched, create an empty syslinux directory in the root of the first partition. If you're adding another live system to a flash drive that already has a working system, the only changes that will be made to syslinux will be the entries added to the boot menu, which you can manually edit. You should not need to reinstall the boot loader. 4. Persistence You can use a separate partition or a loopback file for persistence, which will allow you to save files during a live session. If you want the persistent volume to be encrypted, see the section on ENCRYPTED PERSISTENCE below. PERSISTENT PARTITION You can create the second partition now if you didn't already do it. Use a linux filesystem format give the filesystem a label. The standard method is to label it "persistence" so that the live system can use it when you boot with the "persistence" boot option. If the partition you choose for persistence has a different label, the program will detect that and will place an entry in your boot menu for using this partition with the option persistence-label=LABEL The disk label will also be used in the menu label. After closing or bypassing gparted, choose the partition you want to use. It will probably be the second partition on the selected device. Setting persistence for /home will allow you to save files in your home directory. Full persistence will allow you to save both system files and home files. You can choose to edit persistence.conf yourself if you want different options. Other, non-standard methods for read/write access to a partition are available in the advanced options. PERSISTENT LOOPBACK FILE You can create a loopback file to be used instead of a partition for data storage. This can be set up with or without persistence and with or without encryption. If you put the file on the FIRST PARTITION, you will need to use a patched initrd and boot with the live media in read/write mode. PATCH THE INITRD FIRST, THEN CREATE THE LOOPBACK FILE. That way, the boot menu entry will be generated correctly. Default settings do not require a patched initrd. To change file sizes listed in the loopback settings window, edit /etc/refracta2usb.conf. To change the default settings, edit /usr/lib/refracta2usb/functions_r2u. USE AN EXISTING LOOPBACK FILE FOR PERSISTENCE This creates a boot menu entry for the selected loopback file with the selected live system on the usb. It also adds persistence.conf to the filesystem inside the loopback file. This is useful if you make a loopback file without persistence, and you later want to use it for persistence, or if you want to use the same persistent volume with different live systems (which may have unintended and/or disasterous consequences) or if you replaced one live system with another that does not have the exact same name. The alternative to using this function is to edit the boot menu and persistence.conf manually. ENCRYPTED PERSISTENCE The live iso you use must have cryptsetup installed. After you add an iso or its contents to create a live-usb... Create encrypted, persistent partition or loopback file (not on first partition). If the initrd you choose to use with this volume is not ready for encryption, you'll get a warning, and a script will be copied to the root of the usb device. In Debian or Devuan Jessie: If you don't get that warning, try booting into the live system with persistence. If you get asked for the passphrase to open the volume, then it's working. If not,.. (This worked with Refracta-7.8/wheezy, but syslinux was from jessie) boot back to your installed system and run Patch-initrd on the initrd for the live system. Edit the boot menu entry for the persistent volume to use the patched initrd. Then reboot the live system with persistence (using the patched initrd.) That should be enough. If you do get the warning about the initrd and encryption, there are two ways to deal with this. 1. This works in Debian or Devuan Jessie Reboot into the live system, using the menu entry for persistence. This is not true persistence (you won't be asked for the password to decrypt the volume.) but root will be able to write to the first partition. Run the script (as root) to update the initrd: /lib/live/mount/persistence/sdX1/update-init-crypt.sh (Replace sdX1 with the correct device.) The script ends with nano opening the boot menu. Make sure the boot entry uses the correct name of the rebuilt initrd. 2. This might work in wheezy, and does work in jessie. If you can, boot into the system you used to create the snapshot.iso. Before making a new snapshot, run CRYPTSETUP=y update-initramfs -u Then make a new snapshot. It will be ready for encrypted persistence when you add it to the live-usb. RESCAN This will unmount the usb and clean up some stuff. You may need to run this when you get mount errors after doing several tasks. ______________________ ___ TOOLS ___ - Edit: Edit boot menu on completed USB thumb drive. Opens the boot menu in a text editor for manual changes. - Bootloader: (Re)install syslinux to partition and to mbr. This will (re)install syslinux to the first partition and the Master Boot Record and it will copy some syslinux modules from the host system. If there is no /syslinux folder in the root of the flash drive, one will be created and populated with menu and help files. - Alternate_mbr: Alternate bootloader (install-mbr) This will run 'install-mbr' on the selected drive. Useful if the syslinux bootloader won't work for you. When you reboot with this, you may be presented with a prompt of 'MBR 1FA'. Type the number "1" to boot. (If so, check partition for boot flag.) - Patch_initrd: Rebuild initrd for runtime read/write fat32. (patch-initrd-gui) Select the initrd to patch. You probably want the one that's already on a live-USB, so that's where the file-selection window opens. If you choose a different one, it's up to you to make sure that the matching kernel is present on the usb drive. This allows you to save or edit files in the root of the live media while it is running. You can use this to edit the boot menu and help files, or to save files on a fat32 partition that can be seen by Windows. - Mount_loop and Unmount_loop If you have loopfiles that aren't set for booting with persistence, you can mount/unmount them easily. Hint: If you have a multi-boot usb with separate loopfiles used for persistence with the different live systems on the disk, and you need access to files that are in the loopfile of a system other than the one that's currently running, you can run these to mount and unmount selected loopfiles. - Delete_loop Sometimes, unused loop devices may be left by Mount_loop, especially if the task is stopped before it finishes. This will find the leftover and loop devices and let you delete them. You probably won't need to use this. - File manager Mounts and opens the first partition of the device in a root file manager, in case you want to delete a live system from the usb or check to see what's on it. It's a root file manager. Be careful. ________________ ___ EXAMPLES ___ --------------------------------------------------- Sample Debian-Live Setup: Single-boot (obsolete) --------------------------------------------------- NOTE: There is no longer a /live directory in the root of the device. The system goes into a named directory, as in Multiboot, below. Not complete. There may be other files and directories. |-- live | |-- filesystem.squashfs | |-- initrd.img | `-- vmlinuz |-- syslinux | |-- live.cfg | |-- (other files, including boot help screens and splash image) ---------------------------------- Sample Multiboot setup: (This is the default for refracta2usb-2.0) ---------------------------------- Each operating system is in its own named directory. The isolinux inside each of these named directories is inactive. Only the syslinux in the root of the device is active. |-- refracta_7.2 | |-- live | |-- filesystem.squashfs | |-- hooks | |-- initrd.img | |-- memtest86+.bin | `-- vmlinuz | |-- pkglist_refracta_7.2_i386-20131020_0352 | `-- package_list | |-- Release_Notes | `-- isolinux |-- sid1209 | |-- live | |-- filesystem.squashfs | |-- initrd.custom_sid.img | |-- initrd.img | `-- vmlinuz | `-- isolinux `-- syslinux |-- live.cfg |-- (other files) ------------------------------------ Sample Combined Setup: ------------------------------------ You might get something like this if you start with a stock Debian-live image and turn it into multiboot. |-- live | |-- filesystem.squashfs | |-- initrd.img | |-- memtest86+.bin | `-- vmlinuz |-- sid1209 | |-- live | | |-- filesystem.squashfs | | |-- initrd.custom_sid.img | | |-- initrd.img | | `-- vmlinuz | `-- syslinux |-- sid_mar | |-- initrd.img | |-- sid-refracted_a19-20140330_2152.iso | |-- syslinux | `-- vmlinuz |-- syslinux | |-- live.cfg | |-- (other files) -------------------------------------- Sample multiboot with EFI boot files -------------------------------------- |-- boot | |-- grub | |-- efiboot.img | |-- font.pf2 | |-- grub.cfg | |-- splash.png | `-- x86_64-efi |-- efi | |-- boot | `-- bootx64.efi |-- jessie_ice | |-- initrd.img | |-- jessie-ice-template-20170103_1454.iso | `-- vmlinuz |-- refracta8_amd64 | |-- live | |-- filesystem.squashfs | |-- initrd.img | |-- memtest | `-- vmlinuz | |-- pkglist_refracta8_xfce_amd64_rc1-20160923_1334 | `-- package_list |-- syslinux |-- live.cfg |-- (other files) -------------------------------------- Sample persistence.conf: -------------------------------------- /home union,source=. # Use this if you copied /home/* to the volume labeled "persistence". #/home bind,source=. # Use this for full persistence (entire filesystem writable.) #/ union,source=. --------------------------------------- Sample boot menu entries: --------------------------------------- # default Refracta boot label live menu label Refracta (default) kernel /live/vmlinuz quiet append initrd=/live/initrd.img boot=live # boot with standard debian-live style persistence label persist menu label Refracta (persistent mode) kernel /live/vmlinuz quiet append initrd=/live/initrd.img boot=live persistence # boot with standard debian-live style persistence # but with disk label other than "persistence" # and use network configs saved in snapshot. label persist menu label Refracta (persistent mode) kernel /live/vmlinuz quiet append initrd=/live/initrd.img boot=live ip=frommedia persistence persistence-label=LABE # boot with patched initrd to allow editing of files in root of media label rwmedia menu label Refracta (rw media)) kernel /live/vmlinuz quiet append initrd=/live/initrd.custom.img boot=live mountmode=rw,noatime,umask=000 # multiboot with loopfile on a different partition label loopten menu label Refracta (loopten) kernel /sid/live/vmlinuz quiet append initrd=/sid/live/initrd.img boot=live live-media-path=/sid/live persistence persistence-media=removable-usb persistence-path=/loopfiles/ persistence-label=loopten # multiboot with loopfile on same partition and in same folder, with patched initrd label fakeloop2 menu label Refracta (fakeloop2) kernel /fakesid/live/vmlinuz quiet append initrd=/fakesid/live/initrd.custom.img boot=live mountmode=rw,noatime,umask=000 live-media-path=/fakesid/live persistence persistence-media=removable-usb persistence-path=/fakesid/ persistence-label=fakeloop2 persistence-encryption=none,luks # blacklist nouveau label nomodeset menu label Refracta (no modeset) kernel /live/vmlinuz quiet append initrd=/live/initrd.img boot=live nomodeset nouveau.blacklist=1 ### Boot_ISO bug: (Note: This bug seems to be gone in 0.9.5 and 0.9.6) # Adding a patched initrd or loopback file to a folder that uses an intact iso file to boot # results in an incorrect menu entry. In the example below, the first entry is correct. It was # created when the iso was first added to a folder named 'sid'. But the entries created for initrd.custom.img # and persistence are missing the findiso option, and they have the wrong path for kernel, initrd # and live-media-path. label sid menu label sid kernel /sid/vmlinuz quiet append initrd=/sid/initrd.img boot=live ip=frommedia findiso=/sid/snapshot140102_1452.iso label initrd.custom.img menu label Refracta (initrd.custom.img) kernel /sdc1/live/vmlinuz quiet append initrd=/sdc1/live/initrd.custom.img boot=live ip=frommedia mountmode=rw,noatime,umask=000 live-media-path=/sdc1/live label persistence menu label Refracta (persistence) kernel /sdc1/live/vmlinuz quiet append initrd=/sdc1/live/initrd.custom.img boot=live ip=frommedia mountmode=rw,noatime,umask=000 live-media-path=/sdc1/live persistence persistence-media=removable-usb persistence-path=/sid/ persistence-encryption=none,luks # Here are the corrected entries. # /sdc1/live is replaced with the named folder /sid # and the findiso option is added. label initrd.custom.img menu label sid (initrd.custom.img) kernel /sid/vmlinuz quiet append initrd=/sid/initrd.custom.img boot=live ip=frommedia mountmode=rw,noatime,umask=000 findiso=/sid/snapshot140102_1452.iso label persistence menu label sid (persistence) kernel /sid/vmlinuz quiet append initrd=/sid/initrd.custom.img boot=live ip=frommedia mountmode=rw,noatime,umask=000 findiso=/sid/snapshot140102_1452.iso persistence persistence-media=removable-usb persistence-path=/sid/ persistence-encryption=none,luks