Re: Help needed to do a backup
 

I doubt anyone's going to send you the image, it is where all private stuff is kept... :(

Re: Help needed to do a backup
 

There where a file before but cannot find it anymoore,jolla-sailfish-1.0.8.21.tar.xz
just send a private message If someone know where to get It
Edit..I see It was removed,Jolla care then

Re: Help needed to do a backup
 

It's 2017 now and I'm still using a Jolla 1 phone.

My BTRFS partition got corruption.
(One log entry causes the kernel to segfault while trying to replay).

I made the mistake of trying the recovery mode without much checking.
I ended up with a partition with cleared checksum, that still doesn't mount in-kernel (Still the same replay-log segfault).

Luckily, I can recover nearly every file I need from my busted partition (thank you BTRFS's Copy-on-Write !!!) using btrfs tools which will use whichever copy happens to not be busted.

Now instead of trying to rebuild a functioning BTRFS partition myself....
(which would be possible at this point but tedious - recovery mode lost the checksums, btrfs tools doesn't save the posix access rights, and I still need to fix the segfault causing log entry. So quite some work until phone is up again)
...I wanted to know if anyone happens to have a useable "factory-@" Image file ?

(Jolla Care are slow to answer my Zendesk ticket, and as mentioned above, it's not possible to publish it on web due to copyrighted bit from Qualcomm)

Re: Help needed to do a backup
 

Update:

My laptop's btrfs restore works even better and I got everything I wanted out of the busted partition.

I managed to rebuild a new BTRFS partition with both the recovery bits ( 'factory-@', etc. ) and the regular subvolumes ( '@', '@home' ).

I replaced mmcblk0p28 and now I've got everything back working, not even needing to do a factory reset.

("Copy-on-Write", bitches ! It works)

Re: Help needed to do a backup
 

Argh, I had send in my Jolla for service to Finland.See here.
They actually had to change mainboard because it was completely dead. So I had it back now for three weeks or so,everything set up again, as far as possible since I didn't have a full backup and only an older one from backup function.
Last week I found it on my table stuck on boot screen. This time I could get into recovery mode, but...

I thought, the option in that menu are actually to be used! Naah, almost everytime I tried something, it disconnected telnet. An everytime I tried again, I made it worse. factory-@ was broke pretty early maybe even form the beginning.

Yesterday I finally got around to make my own image file from the remaining pieces, while adding missing pieces from /@ which I thought were part of the original image (mostly translations). It seems, it was not enough to be recovered :( Tar complained about errors while packing and unplacking as well.

I am proud to at least make it to the next stage: It went from bootloop to "trying to revover file system" but ends at "please seek service".

Could someone help me out with advice or even factory-@? factory-@home seems to be OK. I understand, these parts don't contain personal information as with the other 2 subvolumes.

Re: Help needed to do a backup
 

*BUMP*

Sorry to doublepost. I could still use some help here. I already contacted jolla support, but they seem to be busy shipping sailfish X.

Where could I get help from? For a start I would at least try to confirm contents of factory-@ just to be sure of it's completeness.

Re: Help needed to do a backup
 

I managed to build a new image too. There were some files missing from originial, if I found all that is. I added them from my regular linux machine as there were all public files like /zoneinfo folder and xmls in /application.

But:
When I write the new image to mmcblk0p28 everything seems fine, btrfs check gives no errors.
After reboot, I get the "trying to recover" followed by "seek service" I mentioned above.
When doing another btrfs check, file system is now broken again with lots of csum erros and more.

What am I missing?

btw: asking on #jollamobile or zendesk seems to not help. Please help somebody!!

Re: Help needed to do a backup
 

Actually, trying to fsck a BTRFS partition is a VERY BAD idea.
(Same applies for other CoW filesystems like ZFS and log-structured filesystems like UDF and F2FS).

The best is to always simply try to get an early version (before the "copy" part of Copy-on-Write).

For BTRFS, things that might help :
- mounting with "recovery" (will scan back the older copies until it finds the latest usable) (for kernel 4.6 and newer, its now called "usebackuproot")
- mounting with "nologreplay" (completely ignore the journal log)
- using "btrfs restore -s -x -m -S -i " on an image transfered to a laptop.

By write, you mean you DD an image file over the raw device.
Not that you mount the (perhaps corrupted) partition and untar your files inside ?

By check you mean mount and "btrfs scrub start" ?

To me, that looks like a broken btrfs partition.
If you have some place where I can send you the file (FTP server ? Something else ?) I can send you my recovery partition.

I just can't post it publicly, because it contain non-public code (mostly driver and blobs from Qualcomm which make it illegal to post online).

Re: Help needed to do a backup
 

yes, seems I found that out the hard way. On the other hand, I didn't use any options other than what Jolla gave me at hand, which is: recovery menu with options 5, 1 and 4 (used them in that order after each failing) . Reaching No.4 the other two already broke everything.
Thx, will try these. Up until now I did a btrfs restore -ivso (which is btrfs 3.16) from device to sd card.
By write I mean DD ;)
By check I mean unmounted btrfs check (without --repair)
One thing I found out lately: dmesg gives me a unsupported option (100). I guess the btrfs created on my laptop is not compatible with the old 3.16, so it won't mount. In the end this might cause the errors being created on first reboot, when J1 tries to repair automatically?
At the moment I try to create btrfs on device and the restore from dd image file.

I do not have a publicly writable storage for file transfer. Will see, if I can get a ftp server running temporarily.

Appreciate your help!

Re: Help needed to do a backup
 

Yup, definitely.

Format it with (where <uuid> is the UUID of the old partition, or alternatively, let btrfs generate a new UUID and upgrade your /etc/fstab file) : That should do it.

The 3.4.108.20170530.1 kernel onboard of the Jolla 1 is a tiny bit old.
As such it doesn't support some feature that later versions of BTRFS have enabled by default.
If you generate it on a laptop/desktop your BTRFS will be way too modern.
If you generate it on device, the btrfs-tools are a tiny bit more modern too (3.16 instead of 3.4).

Details :

• -n / --nodesize : it used to be 4096 by default on older (<= 3.11), nowadays (>= 3.12) it's 16k
• -O / --features : that the most critical part
  • ^ext-ref : It's default on on 3.16 (e.g.: btrfs-tools on jolla), but it's not available before 3.7 (like on the jolla)
  • ^skinny-metadata : it's default on with 3.18 (like on your desktop) but it's not available before 3.10 (like on the jolla)
• -d / --data and -m / --metadata : although it's not mandatory, it helps a lot on space-constrained devices like the Jolla.
• -L / --label : it's fixed with "sailfish"
• -U / --uuid : Sailfish identifies and mounts the partitions based on UUID. The one of the partition must match the one in /etc/fstab

More information about features in BTRFS :
I'll MP.

Re: Help needed to do a backup
 

Check your private message.

The Sailfish recovery partition normally has 'xz' available.
(So you could pipe xzcat into dd)

At less than 300MB, you should be able to even store it on a FAT32 SD Card or safely send it over USB network cable with netcat.

It's the base I've rebuilt with my "factory-@" bla partitions.
It's formated as stated above (option compatible with Jolla kernel 3.2.108 + using "single" instead of "dup" for metadata).
That's the base out of which I've subsequently done my full restore (I copied all my rescued files on "@", "@home" after snapshotting).

Re: Help needed to do a backup
 

Thx man, will try asap.

I found out more btw: the instructions I was following were for an older btrfs-tools. It should have been
That was the unsupported feature with code (0x100). Found out with
Seems I learned a lot so far about btrfs.

EDIT: LOL, just read your previous post. You explained the same thing.

Re: Help needed to do a backup
 

What version is your snapshot from? 2.1.1.26?

Re: Help needed to do a backup
 

Sadely no.
I've bought my phone back in 2014, so the "factroy-@" is some 1.xx old stuff.
You'll have to jump through several OTA until you reach the current 2.1.2 if you go this route.

Yup.

I would also recommend you to consider two other options, as I've said before :

• --uuid / -U : and you give it the same UUID as the old partition (use blkid command) and as in the /etc/fstab files.
  That will save tons of headache "why isn't the patition automatically mounted" ?
• --metadat / -m : set it to single (instead of the default dup).
  As I've written in the TJC post : dup doesn't make sense on flash media (both writes will often end up grouped together and you lose any benefit from duplication), and it wastes precious space on tiny flash partition.
  single helps a lot against enosp errors

A few key stuff :

• For checking the partition, periodically while mounted run :
  
  Code:

  ---

  btrfs scrub start -B /

  ---

  (-B to run it in foreground). This will scan all the chunks and check that they match their checksum.
  It's a bit more thorough than btrfs check : btrfs check only check if the general structure is coherent, something that the filesystem driver does any way upon mounting (to decide which copy to use). scrub will additionally check the content itself.
  I've put scrub as part of the "check for updates" script that I run every week or two.
• For space management, periodically while mounted run :
  
  Code:

  ---

  btrfs balance start -v -dusage=60 -musage=60

  ---

  This will rewrite data and metadata into new chunks, trying to group more data into less chunks (if two chunks are only 40% full - e.g.: because old deleted copies got enevtually garbage collected - they will be written into a single new chunk filled to 80%)
  The command :
  
  Code:

  ---

  btrs-balancer balance

  ---

  used internally by the installer does the same, but going by incremental steps (which help if there no space to allocate a new chunks) , and all the way to 100% so all the new chunks are filled to the brim and there's as much free space for newer chunks as possible.
  I run the 60% manually as above maybe once per month
• Every now and then, you can run :
  
  Code:

  ---

  fstrim /

  ---

  to tell the flashmedia that it can erase and consider the space free. Du to how copy-on-write works, there's no point in doing it online (no need to mount media with the "discard" option. Doing it every now and then (e.g.: after balancing when there a lot of space that got freed) is enough
• On heavily fragmented files you can run :
  
  Code:

  ---

  btrfs filesystem defrag -v <file>

  ---

  BTRFS is a Copy-on-write system (like ZFS, and a bit similar to log-structured filesystems like UDF and F2FS). It means in never overwrites. When you modify a file, it always write a new copy of this modified part, then updates the pointers to that modification, then eventually garbage collects the old copy if its not in use anymore (e.g.: by an older snapshot).
  That means that, for file that see a lot of small random writes, there's a lot of fragmentation, and to read anything from these, the btrfs driver has to travers a complex maze of pointer to finally reach the most up-to-date copy of the data (meaning a CPU and RAM performance hit). Typically this will happens with the various SQLite databases used everywhere. (You WhatsApp database is a prime example). Doing a defrag on such big SQLite files, will ask btrfs to rewrite the whole file as new continuous single copy, which helps a lot against fragmentation (and gives a small help on the overall phone performance).

Re: Help needed to do a backup
 

OK, so I started the device to update to the version of my backup. There seems to be a problem about network. I can't start WLAN (please reboot for network) and SIM card is not recognized.

Is this because there are blobs incompatible or are there blobs missing at all?

Re: Help needed to do a backup
 

Blobs might be incompatible.

You might want to either check the tutorials out there for manually resetting to factory,
Or see the content of the factory reset script inside the recovery
Or try a factory-reset from the recovery.

I seem to vaguely remember that the factory-reset needed to also flash a few blobs into the special purpose partition (all the other beside mmcblk0p28)

Or you could try to see if you can recover as much as possible using "btrfs restore" (including the file permission, extented attributes, etc.) on your laptop/desktop. It there aren't too many files missing you might as well be able to boot into your latest installation.

Re: Help needed to do a backup
 

Yes, I'm getting close!

I reviewed all the steps while creating a new backup image. And I found a major stupidity of mine: I did btrfs restore into a new btrfs img mounted as loop. But I did not create any subvolumes, so the restored data were only directories. There is no visible difference when doing "ls" on the mounted img.

In the end, my new image was bootable after putting the backed up data into subvolumes.

Now there is on thing left: after boot, I get a white screen. While this tells me, it is booting to a kinda workable state, its not yet usable :(
I chrooted from recovery to unapply patches from patchmanager, as I have read these might be the cause. patchmanager --unapply-all didn't help, still only white screen.

Is there any other way to fix this? I am not on my home network, so the phone is not on the network automatically. Any reinstall has to come from within the phone I guess?

EDIT: on further inspection I found more errors :(
Seems all the btrfs scrub, btrfs check and btrfs restore didn't catch some stale file handles.
So doing ls -laR from the top is gives me the following stale file handles
@/usr/lib/qt5/qml/Sailfish/Store
@/usr/lib/qt5/qml/Sailfish/Lipstick
@/usr/lib/qt5/qml/Sailfish/Email/libsailfishemailplugin.so
@/usr/lib/qt5/qml/Sailfish/Email/qmldir

well, its not in factory-@ so I try to get those files... stay tuned ;)

Re: Help needed to do a backup
 

YES!!!!

I finally can use my phone again. And there is no reinstalling anything. It is just back normal with all my settings and data (well except, that I reverted patchmanager patches).

I will write a summary soon on what was actually needed. There are some catches, if someone (like me) follows older instructions.

Right now I am not in the mood for a writeup, just want to relax a moment and just enjoy my J1 ;)

EDIT: I put the instructions of what I did into another thread here

Re: Help needed to do a backup
 

For the record, if anyone wants to use any such restore :

To flash the blobs that corresponds to some version of Sailfish :

You need to chroot into Sailfish if you're in recovery mode :
Once in Sailfish root, you need to run all the scripts that are in :
(Well, obviously, if you're in recovery, that means you already have a functional recovery, and don't need to run flash-recoveryimg-sbj.sh)

flash-bootimg-sbj.sh : will flash the kernel+initrd into the boot partition.
flash-persist.sh : will flash the content of /persist
flash-platform-images-sbj.sh : will flash all the device-specific blobs into their specific partition (as normally defined in the android standards) : /firmware, /drm, and several other partitions.


If you've chrooted from recovery, remember to exit and umount everything in reverse order.