[XZ1] rooted kernel hiding bootloader unlock with working fota - Sony Xperia XZ1 ROMs, Kernels, Recoveries, & Othe

rooted kernel hiding bootloader unlock
with working sony stock fw fota updates
for Sony Xperia XZ1​
Firmware Over the Air system updates have been disabled/not working with sony xperia phones with unlocked bootloader.
Also many sony drm functions are disabled if fw detects unlocked bootloader even if device master key was recovered.
I've implemented a kernel patch for xperia XZ1 Compact / XZ1 / XZ Premium phones that properly masks bootloader unlock status so it appears as still locked for sony stock firmwares.
This allows FOTA updates to be installed if running completely unmodified stock firmware. This is possible if this kernel is just booted from usb via fastboot instead of flashing it.
The kernel is pre-rooted, so you can have root as usual with magisk when running this kernel (you can use magisk system less patching to make changes to system/vendor partitions without actually modifying them).
For oreo fw the boot process is patched to hide magisk from sony ric daemon that stops the boot in case it thinks the bootloader is still locked. This special patch allows to pass safetynet including cts while having properly working magisk.
This kernel may be used (flashed) just to properly enable sony drm features, like video image enhancements, if device master key was recovered via locked state TA restore.
The bellow described way to install FOTA system update works with both - phone with TA restored and phone with drm keys lost. Both variants have been tested with xz1c.
How to use this kernel while planning to do FOTA system update eventually
Update: please see here for the latest usage instructions for kernels in flashable zip archive.
Please see screenshots bellow for this kernel in action (with xz1c) doing fota system update from oreo to pie and from pie to next pie version. There is also a video documenting this here. Few longer waiting parts have been cut out to fit the video under 15 minutes of youtube limit for not verified accounts.
if your bootloader is still locked
Use renoroot exploit to backup your TA, unlock your bootloader and restore TA-locked to recover device master key as described in
[XZ1c/XZ1/XZp] temp root exploit to backup drm keys implemented thread.
select one of the prepared kernels and download it
make sure you are running unmodified stock firmware
You need the version corresponding to the selected kernel - reflash the firmware to make sure it is unmodified.
Please note: any mount of /system or /vendor partitions in write mode would result with modifications even if nothing is copied there.
Be aware that some zip packages flashed from twrp may mount the partitions for write access even when that is not needed.
reboot the phone to fastboot mode
Use either "adb reboot bootloader" or
enter fastboot by holding powered off phone's volume up key while connecting it to PC via usb cable and use 'fastboot reboot bootloader' command.
boot the downloaded kernel via fastboot
For example (xz1c):
Code:
fastboot boot boot-G8441-47.1.A.16.20-hideunlock-rooted.img
enjoy your rooted phone which thinks it is still locked
Sony apps will be offered to install/update. System FOTA update may come.
Magisk will provide your root when magisk manager app is installed (offered on the first boot).
if you need to use a custom recovery, like TWRP
Do not flash it. If you do, FOTA update verification will fail.
Instead use 'fastboot boot' the same way as with the kernel above, but instead of the kernel, boot the twrp image without flashing it.
to install a FOTA system update
just start the update as usual
let it run until it finishes the installation
try to catch the restart then and hold volume up that time to enter fastboot
you need to use following command to make next boot working
Code:
fastboot reboot bootloader
use 'fastboot boot' to boot kernel for fw to which fota updating to,
for example (xz1c):
Code:
fastboot boot boot-G8441-47.2.A.4.45-hideunlock-rooted.img
if you miss the restart (or do not have the right kernel version),
it does not matter, the installation will finish even when bootloader unlock is detected with the last reboot to updated system,
so just 'fastboot boot' the corresponding 'hideunlock-rooted' kernel then
Alternative use of this kernel
If you do not like booting from usb via fastboot to startup your phone, you can flash the kernel and boot normally.
But if you like to install FOTA system update then, you would need to flash the stock kernel first in order to make the fw untouched again (assuming no other changes to the fw, like system or vendor partitions, have been done) and boot the patched kernel via 'fastboot boot' as described above.
You can backup stock kernel (and recovery) to avoid need to download full stock fw when you need to restore stock kernel & recovery when you decide to install fota system update - see here and following post for more details please.
If you do not care about FOTA, just do not install it.
And use this kernel just to enable all sony drm features that are available on a locked phone (assuming locked state TA has been restored).
In case you like to make some modifications to system or vendor partitions (as you do not care about fota), you would need to disable verity in the kernel - please see post#3 for noverity variants of oreo kernels and linked post describing howto switch verity off via magisk in all pie kernels.
Downloads
See the post#2 please.
Source code
patched kernel sources to hide bootloader unlock (my-bluhide/* branches)
https://github.com/j4nn/sonyxperiadev-kernel-copyleft
patched magisk sources to hide magisk from sony ric daemon on early boot phase (v19.1-manager-v7.1.2-ric branch)
https://github.com/j4nn/Magisk/tree/v19.1-manager-v7.1.2-ric
The patches are provided under GPL (that means you may include them in your builds, but you need to provide buildable source of released binaries /true for any kernel change btw/).
Credits
Thanks to @tonsofquestions for lot of initial testing of this concept when I did not have a phone with unlocked bootloader and for discovering the need to reboot to fastboot by a command to make the 'fastboot boot' command properly boot the supplied kernel image.
Thanks to @topjohnwu for his excelent magisk tool.
If you find my work useful, consider donating here please:
https://j4nn.github.io/donate/
Thank you.
XDA:DevDB Information
kernel_bluhide_poplar, Kernel for the Sony Xperia XZ1
Contributors
j4nn
Source Code: https://github.com/j4nn/sonyxperiadev-kernel-copyleft
Kernel Special Features: proper hiding of bootloader unlock, sony ric with magisk hack
Version Information
Status: Stable
Stable Release Date: 2019-02-10
Created 2019-02-10
Last Updated 2019-08-07

---

Downloads
- hideunlock kernel pre-rooted boot images:
Xperia XZ1 (G8341/G8343)
boot-G8341-47.1.A.16.20-hideunlock-magisk-19.1.img
boot-G8341-47.2.A.4.45-hideunlock-rooted.img
boot-G8341-47.2.A.6.30-hideunlock-rooted.img
boot-G8341-47.2.A.8.24-hideunlock-rooted.img
boot-G8341-47.2.A.10.28-hideunlock-rooted.img
boot-G8341-47.2.A.10.45-hideunlock-rooted.img
boot-G8341-47.2.A.10.62-hideunlock-magisk-19.3.img
Xperia XZ1 Dual (G8342)
boot-G8342-47.1.A.16.20-hideunlock-magisk-19.1.img
boot-G8342-47.2.A.4.45-hideunlock-rooted.img
boot-G8342-47.2.A.6.30-hideunlock-rooted.img
boot-G8342-47.2.A.8.24-hideunlock-rooted.img
boot-G8342-47.2.A.10.28-hideunlock-rooted.img
boot-G8342-47.2.A.10.45-hideunlock-rooted.img
boot-G8342-47.2.A.10.62-hideunlock-magisk-19.3.img
- hideunlock kernels flashable to multi fw versions (see here for usage howto):
Xperia XZ1 (G8341/G8343)
kernel-G8341-47.1.A.16.20-hideunlock.zip
kernel-G8341-47.2.A.10.62-hideunlock.zip
kernel-G8341-47.2.A.10.80-hideunlock.zip
kernel-G8341-47.2.A.10.107-hideunlock.zip
kernel-G8341-47.2.A.11.228-hideunlock.zip
Xperia XZ1 Dual (G8342)
kernel-G8342-47.1.A.16.20-hideunlock.zip
kernel-G8342-47.2.A.10.62-hideunlock.zip
kernel-G8342-47.2.A.10.80-hideunlock.zip
kernel-G8342-47.2.A.10.107-hideunlock.zip
kernel-G8142-47.2.A.11.228-hideunlock.zip
Screenshots of XZ1c FOTA system update from oreo 47.1.A.16.20 to pie 47.2.A.4.45
(video available here)

---

Downloads
This is for alternative use only - please see post#10 for more details.
boot-G8341-47.1.A.16.20-hideunlock-magisk-19.1-noverity.img
boot-G8342-47.1.A.16.20-hideunlock-magisk-19.1-noverity.img
Screenshots of XZ1c FOTA system update from pie 47.2.A.4.45 to pie 47.2.A.6.30 version
(video available here since 08:10 time)

---

This should be the first paragraph!
j4nn said:
[...] use this kernel just to enable all sony drm features that are available on a locked phone (assuming locked state TA has been restored).
Click to expand...
Click to collapse
Since I do not care about OTA updates, because we can download every firmware via XperiFirm and flash it via newflasher, I will use your modded kernel in the 'alternative' way! :good:
j4nn said:
Alternative use of this kernel
If you do not like booting from usb via fastboot to startup your phone, you can flash the kernel and boot normally.
Click to expand...
Click to collapse

---

@j4nn - I flashed "boot-G8341-47.2.A.6.30-hideunlock-rooted.img" on my G8341 which has the latest Pie firmware (G8341_47.2.A.6.30_Customized DE_1310-4290_R6C) installed, but the device did not boot into Android after that! :crying: I flashed the latest official Magisk before your kernel. Do I need to use your patched Magisk instead?
Did you forgot to add the patched magisk flashable img file? "hide magisk from sony ric daemon on early boot phase (v18.1-manager-v7.0.0-ric branch)"
j4nn said:
Downloads
Click to expand...
Click to collapse
Maybe the firmware version is not compatible with the one you took the kernel from? The firmware folder includes:
kernel_X-FLASH-ALL-C93B.sin
boot/bootloader_X_BOOT_MSM8998_LA2_0_P_107_X-FLASH-ALL-C93B.sin
...
Click to expand...
Click to collapse

---

@SGH-i200, I just compared the stock kernel boot image extracted from G8341_Customized DE_1310-4290_47.2.A.6.30_R6C (which I have used to create it) and it seems all good to me:
- os version and patch level match
- device tree blobs are the same
- kernel command line is the same
- extracted kernel config is (except few comments as it is usual) the same, with poplar target (i.e. single sim XZ1)
- ramdisk is the same except changes that are expected (integrated magisk in order to provide root when booted from usb having unmodified stock fw)
Did it result with an infinite hang during boot (which phase of the boot splash animation)?
Or did it reboot during boot (a boot-loop) - which phase?
I am sorry I forgot to mention the kernels for other targets than xz1c have not been tested (I have only xz1c for testing).
But it should be safe to test the above howto as described. Even if the kernel was flashed (instead of just fastboot booted), recovering from this should be as simple flashing stock kernel back.
Anybody else tested this with XZ1?

---

j4nn said:
Did it result with an infinite hang during boot (which phase of the boot splash animation)?
Or did it reboot during boot (a boot-loop) - which phase?
Click to expand...
Click to collapse
The SONY white splash screen appeared (with only black SONY text) and then it hang forever.
j4nn said:
Even if the kernel was flashed (instead of just fastboot booted), recovering from this should be as simple flashing stock kernel back.
Click to expand...
Click to collapse
I flashed the kernel_X-FLASH-ALL-C93B.sin (and everything else as I did to upgrade to latest Pie) from the stock fw to recover.
After the XZ1 was booting again, I executed the fastboot booting, without problems!
adb reboot bootloader
fastboot boot boot-G8341-47.2.A.6.30-hideunlock-rooted.img
Click to expand...
Click to collapse

---

@SGH-i200, that sounds good. Now just flash it:
Code:
adb reboot bootloader
fastboot flash boot boot-G8341-47.2.A.6.30-hideunlock-rooted.img
disconnect from usb and power on.
If it booted from usb, I believe it should boot just fine if flashed as above too.

---

j4nn said:
If it booted from usb, I believe it should boot just fine if flashed as above too.
Click to expand...
Click to collapse
I flashed your patched kernel again and got into a bootloop: Bootloader unlocked warning and white SONY splash screen in an endless loop.
Since my XZ1 was connected to my Windows workstation already, I simply long pressed the volume up button till the notification light went blue, and booted into twrp to restore the stock kernel (rooted by Magisk to remove the dmverity stuff).

---

@SGH-i200, you are right, it is the verity thing.
I've assumed that verity is ignored when kernel detects unlocked bootloader, because the dm-android-verity kernel component used that is_unlocked() call to:
/* Allow invalid metadata when the device is unlocked */
- that comment is in the dm-android-verity.c source code.
But it looks like that is only a corner case and if verity metadata is valid, verity is active (if not disabled in device tree) even with unlocked bootloader.
Now I am wondering: for OTA updates we need untouched system, so having verity enabled in the kernel is good for this purpose - when you flash something that writes to system or vendor, you may detect it soon enough to be able to tell what it was. The main feature of magisk is that you should be able to "modify" system "systemlessly", i.e. without really writing to system or vendor partitions.
From this point of view it should be all good then.
But I understand that the "alternative" use of this kernel, i.e. not planning to do FOTA at all, just using only the unlock hide feature restoring drm functionality and using root the old way (with real writes to system) may be liked.
For this, just use magisk manager main screen, in "Advanced Settings" unselect the "Preserve AVB 2.0/dm-verity" option and then use the "Magisk INSTALL" button, confirm install of Magisk-v*.zip, select "Patch Boot Image File", browse to sdcard where you put my kernel image and select it.
Magisk will repack the boot image disabling dm-verity in device tree blobs, noting where you can find patched_boot.img.
Flash that and you should be good to go.
This procedure is valid for all my pie kernel builds.
I will repack the oreo 16.20 build as using magisk manager gui would not keep the magisk patch for sony ric daemon.
-- edit --
Updated post#3 with downloads of oreo kernels noverity repacks.

---

Thanks for the kernel but after flashing the kernel the phone don't go into doze (deep sleep). Any suggestions on this?

---

j4nn said:
Now I am wondering: for OTA updates we need untouched system, so having verity enabled in the kernel is good for this purpose - when you flash something that writes to system or vendor, you may detect it soon enough to be able to tell what it was. The main feature of magisk is that you should be able to "modify" system "systemlessly", i.e. without really writing to system or vendor partitions. From this point of view it should be all good then.
Click to expand...
Click to collapse
I use AdAway and activated NightLight by copying a apk to /vendor/overlay .
Thanks for creating the noverity Oreo kernel versions! :good:
---------- Post added at 09:04 AM ---------- Previous post was at 08:57 AM ----------
j4nn said:
The main feature of magisk is that you should be able to "modify" system "systemlessly", i.e. without really writing to system or vendor partitions.
But I understand that the "alternative" use of this kernel, i.e. not planning to do FOTA at all, just using only the unlock hide feature restoring drm functionality and using root the old way (with real writes to system) may be liked.
For this, just use magisk manager main screen, in "Advanced Settings" unselect the "Preserve AVB 2.0/dm-verity" option and then use the "Magisk INSTALL" button, confirm install of Magisk-v*.zip, select "Patch Boot Image File", browse to sdcard where you put my kernel image and select it.
Magisk will repack the boot image disabling dm-verity in device tree blobs, noting where you can find patched_boot.img. Flash that and you should be good to go. This procedure is valid for all my pie kernel builds.
Click to expand...
Click to collapse
If I flash your patched Pie kernel via TWRP and flash Magisk right after, this should have the same effect as patching your kernel via the Magisk app, right?
@j4nn - "using root the old way (with real writes to system)" - will flashing Magisk via TWRP change this to system-less root?

---

sinkoo1979 said:
Thanks for the kernel but after flashing the kernel the phone don't go into doze (deep sleep). Any suggestions on this?
Click to expand...
Click to collapse
I have no idea. In my opinion, if you get such behaviour with my kernel, you will get exactly the same behaviour with stock kernel of the same version.
My kernel is compiled from sony's open source kernel package, with kernel config that's identical (except few comment lines) to the one compiled in stock kernel.
My kernel patch only fixes kernel command line that comes from bootloader and hijacks trust zone api to mask the bootloader status to appear as still locked to firmware's userspace.
These changes hardly could have any influence on power saving behaviour.
Please flash the same version stock kernel and test again for deep sleep.
Could not be that caused by some app you've installed?
SGH-i200 said:
If I flash your patched Pie kernel via TWRP and flash Magisk right after, this should have the same effect as patching your kernel via the Magisk app, right?
@j4nn - "using root the old way (with real writes to system)" - will flashing Magisk via TWRP change this to system-less root?
Click to expand...
Click to collapse
Most likely flashing magisk via twrp after flashing the patched pie kernel would disable verity too.
Using the image file - you have it under your control.
Flashing from twrp - some magisk scripting tries to detect if verity should be disabled or not.
Also a kernel image already containing magisk contains the setting of verity and encryption - not sure how that is used when flashing magisk again over it.
I tried it with my xz1c and verity was flipped from enabled to disabled - so the same effect (this time).
But as described, some detection is used, so the results might not always be the same.

---

j4nn said:
I have no idea. In my opinion, if you get such behaviour with my kernel, you will get exactly the same behaviour with stock kernel of the same version.
My kernel is compiled from sony's open source kernel package, with kernel config that's identical (except few comment lines) to the one compiled in stock kernel.
My kernel patch only fixes kernel command line that comes from bootloader and hijacks trust zone api to mask the bootloader status to appear as still locked to firmware's userspace.
These changes hardly could have any influence on power saving behaviour.
Please flash the same version stock kernel and test again for deep sleep.
Could not be that caused by some app you've installed?
reflashed stock firmware and everything is fine. Thanks for the kernel.
Click to expand...
Click to collapse

---

j4nn said:
Downloads
This is for alternative use only - please see post#10 for more details.
boot-G8341-47.1.A.16.20-hideunlock-rooted-noverity.img
boot-G8342-47.1.A.16.20-hideunlock-rooted-noverity.img
Screenshots of XZ1c FOTA system update from pie 47.2.A.4.45 to pie 47.2.A.6.30 version
(video available here since 08:10 time)
Click to expand...
Click to collapse
Hi j4nn
I unlocked my bootloader on 47.1.A.16.20 and lost my drm keys. I upgraded via newflasher_v13 to 47.2.A.4.45 Customised SG as that is where i am based and where my phone is from. Unfortunately when I flash boot-G8342-47.2.A.4.45-hideunlock-rooted.img my phone still boots up showing the device has been unlocked and the Backup and Restore feature of Xperia devices does not work. Am I correct to say if I lost my drm keys that I am unable to use this kernel to hide my bootloader unlock?
Thank you for answering me.

---

Working fine on xz1 pie 47.2.A.6.30 with blx firmware, Kernel flashed with twrp, widevine is L1 now, and Bravia engine working

---

@leonaheidern, the kernel would hide the unlocked bootloader regardless you lost drm keys or restored locked TA.
Please check you kernel build date when you boot your phone in about menu.

---

@j4nn, I have a quick question and a bit off topic, I'm still tinkering a bit but I did manage to backup my ta. and I flashed janjan's so things may seem a bit off, (see pic).. but I did restore my ta. before flashing Jan's kernel, any idea what may happen if I flashed yours on top?
oh and everything is working..

---

@lilbrat, I guess that the result would be the same if you flashed my kernel without flashing janjan's before it.
Just note please, my kernels as they are have verity enabled, so you need to have unmodified stock fw.
Or disable verity as hinted for alternative use.
With my kernel and restored TA your security screen would look perfect as with a still locked phone.

---

j4nn said:
@lilbrat, I guess that the result would be the same if you flashed my kernel without flashing janjan's before it.
Just note please, my kernels as they are have verity enabled, so you need to have unmodified stock fw.
Or disable verity as hinted for alternative use.
With my kernel and restored TA your security screen would look perfect as with a still locked phone.
Click to expand...
Click to collapse
OK.. thanks for the info... now I have another question, now that xz1 has a new update out and granted it will take you a bit to catch up if you need to redo all your kernels you have going... (great work by the way) Will I need to reflash my ta. backup for the new firmware and or just your kernel?