OK, I am starting this thread for a general discussion related to some development work I am currently doing. It involves some fairly low level components of the Android architecture.
I am basing this on CM9, CM being my preferred flavour of Android, but it applies to other flavours too. It will be developed first for an Allwinner A10 tablet (A Momo9 clone) as this is the least important Android device I have lying around, the one which I am least bothered about breaking, but I plan to make these updates as generic as possible to allow easy porting.
OK, on to my objectives:
Make a flexible boot environment system, allowing a device to be booted into different ROMs/configurations, using LVM as a base. This would also allow more flexible use of internal memory, snapshots, and non-persistent data configurations.
*EDIT* By the way, the first part of this (getting LVM2 onto Android) has been completed already by steven676 (see his github repo)
Make additions to CWM recovery to allow management of LVM volumes and boot environments. I have no idea where to start on this one, seeing as I can't even find the source for CWM, but it is necessary to make the project work well.
Make additions to vold and the Android storage settings to allow management of volumes and boot environments from within Android, and to recognise logical volumes for use as internal "SD card". This will probably have to be accomplished using an additional daemon due to license incompatibilities between vold (Apache) and LVM (GPL), but I would like the main interface to my work to be within vold as it is the logical place.
The idea would be that you create an LVM volume group using your internal storage (probably system, data, cache and internal SD partitions), and then define your sys/data/cache/sd etc. as logical volumes within this group. As stated above in objective 1, this gives you flexibility (you can size them as you wish) and access to the snapshot functionality. Snapshots would be great in testing software (you can easily get your device back to a known good config), and would also allow a non-persistent configuration (all changes discarded on reboot, great for handing a device to another person e.g. your kids).
The reason I wish to do this is as a base for my next project, which needs the kind of flexibility this will bring, but I want to keep this discussion on track, so I wont muddy the waters with the details of it yet. However, it will also aid those with small system partitions, developers, and those with small children (or partners... ) who install loadsa rubbish and screw up your device.
The mods I plan to make in CWM should also make it much easier to make update packages for different devices, as I plan to include methods for the updater-script to determine where things should be going (where the system/data/boot etc. devices live), rather than having to hard code them in the script.
So: Thoughts? Advice? Please don't just leave comments of "This is a good/bad idea", I am looking for a constructive discussion.
Thanks in advance
Mouse
I'm not quite following you. Are you planing to boot from an LVM volume? If so, you'll more or less require GRUB2, and does GRUB2 support ATAGS? I believe you'll need 'em both for this to succeed.
Hi
Not quite.
The plan is to use the standard Android boot mechanism to load the kernel and initrd. Within this initrd, the mounting of filesystems (system/data/cache etc) is moved into a separate daemon which chooses them from LVM (or anywhere really). This is limitted, of course, to using just one kernel & initrd (unless kexec can be used, which is beyond the scope of what I need for this project), but it allows much greater flexibility.
drmouse81 said:
The plan is to use the standard Android boot mechanism to load the kernel and initrd. Within this initrd, the mounting of filesystems (system/data/cache etc) is moved into a separate daemon which chooses them from LVM (or anywhere really). This is limitted, of course, to using just one kernel & initrd (unless kexec can be used, which is beyond the scope of what I need for this project), but it allows much greater flexibility.
Click to expand...
Click to collapse
Ah, ok, you are making it easy for you. I do something similar myself. After I boot my custom initramfs, it performs a switchroot to the SDcard, continuing the execution of the real init there. This gives about the same flexibility you are seeking, but also with the same limitation the kernel must stay the same for all systems. This way I can change the system by replacing the SDcard, without the need of LVM.
Yes, this is very similar to what I am in the process of doing. I just think that, especially in devices with large amounts of internal storage (my tab has 16GB) as well as an SD card slot, LVM is a much more flexible solution. Snapshots are the main benefit, but I also plan to use LVM for another project.
Really, the LVM part is pretty simple. The project mentioned on github has the majority of the work done, I am simply extending it at the moment to include some additional features. The majority of the work involved here will be integrating it more closely with both Android and CWM (I expect a steep learing curve: I already develop in both C/C++ and Java, but am not very familliar with the internal workings of Android, and can't even find the sources for CWM).
I do believe it is worth the effort, however. Who knows, it may even make it into CM, or even official Android, at some point... Doubt it, but it is possible
I've created a single primary Ext2 partition in my phone's external SD card but Android refuses to mount it automatically. Whenever I attempted to mount it manually it kept throwing the error "mount operation not supported on transport endpoint".
How can I mount it?
Using (SlimKat) Android 4.4.2.
EDIT: I'm now able to mount it only manually but have to specify ext4 as its filesystem -- why and will it make a difference since ext2 is non-journalled? Also, I tried adding a mount entry in /fstab.smdk4x12 but it was deleted upon reboot; does this mean no manual entries are allowed in that file and I will instead have to hack my own /init-xx.rc file to manually mount the partition at boot time?
miguelg_ said:
I've created a single primary Ext2 partition in my phone's external SD card but Android refuses to mount it automatically. Whenever I attempted to mount it manually it kept throwing the error "mount operation not supported on transport endpoint".
How can I mount it?
Using (SlimKat) Android 4.4.2.
EDIT: I'm now able to mount it only manually but have to specify ext4 as its filesystem -- why and will it make a difference since ext2 is non-journalled? Also, I tried adding a mount entry in /fstab.smdk4x12 but it was deleted upon reboot; does this mean no manual entries are allowed in that file and I will instead have to hack my own /init-xx.rc file to manually mount the partition at boot time?
Click to expand...
Click to collapse
Why not just reformat to ext4?
es0tericcha0s said:
Why not just reformat to ext4?
Click to expand...
Click to collapse
Ultimately that's what I'll need to do but was hoping to use the non-journalled ext2. Is it not supported by Android or it the case that only some versions support it (in which case, what idiocy!)?
Have to say I'm beginning to truly hate Android. They might as well build their own kernel such that Linux (and by implication UNIX) is removed from the mix for they have completely butchered this OS. I suppose this is what happens when egos larger than the world are responsible for designing software; NIH syndrome.
Answering myself: if anyone is wondering how to automount a partition at boot time, you'll have to create your own init script and place in /system/etc/init.d/.
Well, realistically, the amount of people that have any use for mounting ext2/3/4 etc partitions is a very small % of users. Most people with android phones don't even know what Linux is, much less know about different kinds of partitions and what they are used for - or have a need for it. 99% of things you would NEED to do on a phone would be covered by the fat32 and exFat types. Of course, here on XDA, you'll find plenty of posts, guides, complaints about it, etc but there's obviously a certain type of user that seeks out or finds XDA and are more inclined to know of or have use for more technical things like this.
As far as auto-mounting the script on boot, you have to be rooted with init.d enabled and not all phones have full /system RW capabilities to even add stuff like that even when rooted. This is rare, but there's some HTCs and others like that. Often times there are ways around, but just saying, it's not a universal thing.
es0tericcha0s said:
Well, realistically, the amount of people that have any use for mounting ext2/3/4 etc partitions is a very small % of users. Most people with android phones don't even know what Linux is, much less know about different kinds of partitions and what they are used for - or have a need for it. 99% of things you would NEED to do on a phone would be covered by the fat32 and exFat types. Of course, here on XDA, you'll find plenty of posts, guides, complaints about it, etc but there's obviously a certain type of user that seeks out or finds XDA and are more inclined to know of or have use for more technical things like this.
Click to expand...
Click to collapse
That doesn't excuse the fact that they've deliberately crippled the OS. As an example, the FAT filesystems don't support symbolic links, which means that if you want to move any data outside of internal storage for whatever reason, you pretty much need an extX partition. Besides, those people (the vast majority) who don't know and don't care about the internals of their devices aren't the ones creating software for said devices in the first place. We are. And so these technical aspects matter and are relevant to us, not the masses.
es0tericcha0s said:
As far as auto-mounting the script on boot, you have to be rooted with init.d enabled and not all phones have full /system RW capabilities to even add stuff like that even when rooted. This is rare, but there's some HTCs and others like that. Often times there are ways around, but just saying, it's not a universal thing.
Click to expand...
Click to collapse
Didn't know about that limitation. Can you not remount the rootfs with RW privileges? And do you mean to say that some devices don't even support init.d; if so, what mechanism do they have in place?
miguelg_ said:
That doesn't excuse the fact that they've deliberately crippled the OS. As an example, the FAT filesystems don't support symbolic links, which means that if you want to move any data outside of internal storage for whatever reason, you pretty much need an extX partition. Besides, those people (the vast majority) who don't know and don't care about the internals of their devices aren't the ones creating software for said devices in the first place. We are. And so these technical aspects matter and are relevant to us, not the masses.
Didn't know about that limitation. Can you not remount the rootfs with RW privileges? And do you mean to say that some devices don't even support init.d; if so, what mechanism do they have in place?
Click to expand...
Click to collapse
Well, cripple might be a bit of hyperbole considering it's not something most people would need. I get your point though, it is weird that it's not native since it works with Linux generally. You can link symbolically to FAT systems while rooted with something like this:
https://play.google.com/store/apps/details?id=com.devasque.fmount
And yes, the tech people are creating software for these devices, but they are made for the general public, because that's who buy 90% of these things.
Could you please clarify what you said earlier on the read-only init.d and even some devices not supporting it? Again, thanks for your input, es0tericcha0s.
miguelg_ said:
Could you please clarify what you said earlier on the read-only init.d and even some devices not supporting it? Again, thanks for your input, es0tericcha0s.
Click to expand...
Click to collapse
Sure. Most android devices, actually I can't think of any of the top of my head, don't come with native init.d support or even have init.d that is not accessible. It's just not there. It's enabled in almost all custom roms, or you can add it yourself to many stock roms via a couple different ways like this:
https://play.google.com/store/apps/details?id=com.androguide.universal.init.d
As far as the system RW issue, some phones, like many newer HTCs, have the system protected so that you can make changes to the /system while booted, no problem, but once you reboot, all the changes will get undone. Very annoying. Example:
https://www.youtube.com/watch?v=KV3YaMBnEYI
This is a cross-post from a reddit thread I started, but this is probably a more appropriate location for it.
I have been trying to modify files in the system folder for the Android container on the Asus Flip so I can install SuperSu, but have run into some problems.
The system folder is contained in a squashfs image on the chromebook at /opt/google/containers/android/system.raw.img. Mounted squashfs images appear to not support read-write access. I have been able to unsquash the image, add the SuperSU apk to the /system/priv-app folder and su to the /system/xbin folder, and remake the image. This boots, but SuperSU force closes as soon as it starts.
To make tinkering easier, I've tried building a writable image using dd and mkfs. I placed it in a location that has rw access and modified the /etc/init/android-ureadahead.conf script which mounts it to enable rw access. Unfortunately though it won't boot. The boot logs for the android container show a litany of SELinux errors for different things that it could not set context, operation not permitted. I can post the exact log if necessary. Some googling led me to find that the SELinux security context attributes weren't being replicated in my image, so I tried mounting with context and fscontext options equal to the contexts from the original image, but I get the same problem.
If anyone has any ideas I'd be especially grateful.
lionclaw said:
This is a cross-post from a reddit thread I started, but this is probably a more appropriate location for it.
I have been trying to modify files in the system folder for the Android container on the Asus Flip so I can install SuperSu, but have run into some problems.
The system folder is contained in a squashfs image on the chromebook at /opt/google/containers/android/system.raw.img. Mounted squashfs images appear to not support read-write access. I have been able to unsquash the image, add the SuperSU apk to the /system/priv-app folder and su to the /system/xbin folder, and remake the image. This boots, but SuperSU force closes as soon as it starts.
To make tinkering easier, I've tried building a writable image using dd and mkfs. I placed it in a location that has rw access and modified the /etc/init/android-ureadahead.conf script which mounts it to enable rw access. Unfortunately though it won't boot. The boot logs for the android container show a litany of SELinux errors for different things that it could not set context, operation not permitted. I can post the exact log if necessary. Some googling led me to find that the SELinux security context attributes weren't being replicated in my image, so I tried mounting with context and fscontext options equal to the contexts from the original image, but I get the same problem.
If anyone has any ideas I'd be especially grateful.
Click to expand...
Click to collapse
Wayyyy out of my area of expertise, but here's my (completely novice) best guess.
>All Chromebooks are write-protected with a screw on the motherboard
>Putting a Chromebook in developer mode allows for some tinkering ie things like chroots, and on the asus flip, the ability to install apks from unknown sources.
>Unscrewing the write-protect screw allows for the ability to completely install a new operating system or dual boot setup.
>Maybe you need to do that before you're able to accomplish root access?
My other idea would be to try and figure out a way of doing a systemless root?
Also, total aside but since this is the only thread I've found on XDA about this device, I think chroots are theoretically possible now without the need to be in developer mode via Android apps (even without root on Android). Download the GIMP port from the Play Store to see what I'm talking about. Playing around with that for a few minutes really made me wish that it didn't use emulated mouse/keyboard in it's implementation. Also, it appears that apt-get is broken, but regardless it might interest someone out there looking for a project.
back from the dead, any progress on this?
I have been able to successfully root the Android image on my Asus Flip.
I built a blank image with dd in /usr/local, formatted it with mkfs, mounted it to a folder, mounted the original system.raw.img to a folder, copied the files across, placed *all* the SuperSU files listed as 'required' in the SuperSU update-binary in the relevant places in /system in my new image, set permissions & contexts for those files, edited arc-system-mount.conf and arc-ureadahead.conf to point to the new image and, finally, patched /etc/selinux/arc/policy/policy.30 with the SuperSU sepolicy patching tool in order to boot my rooted Android instance with selinux set to enforcing.
I have created a couple of scripts which more-or-less fully automate this procedure, which can be downloaded from nolirium.blogspot.com. Please feel free to download, open the scripts in a text editor to check them out, and try them out if you like. Only tested on Asus Flip, though.
I seem to be unable to post attachments at the moment so I will just add the descriptions here, I could probably post the entire scripts here too if anyone wants. Feel free to let me know what you think.
DESCRIPTIONS:
1-3.sh
Combines the first three scripts listed below.
01Makecontainer.sh
Creates an 900MB filesystem image in /usr/local/Android_Images, formats it, then copies Android system files therein.
02Editconf.sh
Modifies two system files: arc-system-mount.conf - changing the mount-as-read-only flag and replacing the Android system image location with a new location; and arc-ureadahead.conf - again replacing the Android system image location. Originals are renamed .old - copies of which are also placed in /usr/local/Backup.
03Androidroot.sh
Mounts the previously created Android filesystem image to a folder, and copies SuperSU files to the mounted image as specified in the SuperSU update-binary.
04SEpatch.sh
Copies an SELinux policy file found at /etc/selinux/arc/policy/policy.30 to the Downloads folder, opens an Android root shell for the SuperSU policy patching command to be entered, then copies the patched policy back to the original location. A copy of the original policy.30 is saved at /etc/selinux/arc/policy/policy.30.old and /usr/local/Backup/policy.30.old
Uninstall.sh
Removes the folder /usr/local/Android_Images and attempts to restore the modified system files arc-system-mount.conf and arc-ureadahead.conf.
ok so two questions, one do you think this would work on the Acer r13 convertable? and 2 where can I find the actual instructions/scripts
keithkaaos said:
ok so two questions, one do you think this would work on the Acer r13 convertable? and 2 where can I find the actual instructions/scripts
Click to expand...
Click to collapse
The R13 has a 64-bit Mediatek processor, right?
I have added a version for ARM64, but I haven't tested it.
You can find the instructions and scripts at nolirium.blogspot.com
ya, its a mediatek. and thanks ill go see if i can find it
---------- Post added at 03:31 AM ---------- Previous post was at 02:58 AM ----------
wow, ok. i can do this but im not sure i want to.. after reading the possible problems i may run into. Im going to be getting the G. Home in a couple weeks and i gotta keep things running smooth. This seems like going a tad too far then i need to. The other day i had action launcher going and it looked pretty damn good but i really want to try and get the action3.apk that i have put into the pri-app folder or whatever the chromebook uses i found the syst folder but cant access it. Im wondering if i make the machine writable it would work but im afraid of losing my updates, as long as i could do them manualy, i guess that would be cool. Also since im already going on... has anyone found a way to disable the dev boot screen without tinkering with the physical chromebook yet?
SuperSU on Chromebook
Hey there I love this post but unfortunately im on the mediatek (well not unfortunately cause i love it) but i do really want super su .. But i found this other post that i tried out but i am having a problem executing the scripts. When i go to run the first one, it says can not open "name of script" but the dev takes a pretty cool approach. Im still new to Chrome OS but thanks for the post and if you have any advice on executing scripts id love to hear it!! http://nolirium.blogspot.com/
I'm guessing the above post was moved from another thread...
Anyway, it turns out that zipping/unzipping the files in Chrome OS's file manager sets all the permissions to read-only. Apologies! sudo chmod+x *scriptname* should fix it...
Regarding OS updates, I actually haven't had a problem receiving auto-updates with software write-protect switched off; the main possible potential issue I could imagine arising from the procedure I outlined would involve restoring the original conf files if both sets of backups get deleted/overwritten. This seems unlikely, but in that case either manually editing the files to insert the original string (/opt/google/containers/android/system.raw.img), or doing a powerwash with forced update might be necessary in order to get the original Android container booting again.
I don't think anyone's found a way to shorten/disable the dev boot screen without removing the hardware write-protect screw - from what I've read, the flags are set in a part of the firmware which is essentially read-only unless the screw is removed. Perhaps at some point the Chrome OS devs will get fed up of reading reports from users whose relatives accidentally reset the device by pressing spacebar, and change the setup. Here's hoping.
Hey just jumpig in the thread right quick to see if these instructions are old or what-- got a chromebook pro and the notion of having to update a squashed filesystem every timeto install su seems like a pain..
Is there any kind of authoritative documentation/breakdown regarding what Chromeos is mounting where before I start breaking things? Also anyone happen to know if there's a write-protect screw anywhere in the chromebook plus/pro?
Other questions:
* adbd is running, but is not accessible from adb in the (linux) shell, which shows no devices. Do I need to access adb from another device (i'm short a usb c cable right now) or can I use adb (which is there!) on the chrome side to access adbd on the android side?
* Anyone know if adb via tcp/ip is available? Don't see it in the android settings.
Hey,
There's no real documentation AFAIK, the thing is that ARC++ is a bit of a moving target, as it's so actively being developed/reworked. For instance, with the method described earlier in the thread - it started off being possible to just swap out a file location in arc-ureadahead.conf, then they changed it to arc-setup-conf, and now, since a few CrOS versions ago, the rootfs squashfs image is mounted in a loop fashion via the /usr/sbin/arc-setup binary instead, making an overview of the setup somewhat opaque to the casual observer.
I was kind of hoping to implement a kind of hybrid systemless root style setup myself, but unfortunately I haven't really managed to find the time to sit down and fully figure out a few parts of the puzzle, in particular relating to minijail and working with namespaces. So, I'm still using the method mentioned in posts above for my rooting needs at the moment, the only significant changes being that at the moment I'm replacing /opt/google/containers.android.system.raw.img with a symlink to my writeable rooted rootfs img, and also that in recent CrOS versions the mount-as-read only and debuggable flags can be found in /etc/init/arc-setup-env ("Environment variables for /usr/sbin/arc-setup").
In general though, one can kind of get an idea of what's going on in the default setup by reading through the various /etc/init/arc-* Chrome OS upstart jobs (and their logs in /var/log). Though, like I say, things keep changing around somewhat with every CrOS update, as the implementation 'improves'. As time goes by, and the subsystem matures, it'll certainly be interesting to see what other approaches are possible relating to customizing Android on Chrome OS.
There should definitely be a write protect screw somewhere on the motherboard for the Samsungs, but so far I haven't come across any pics showing exactly which screw it is. So far, no-one seems to have been brave/foolhardy enough to fully tear down their own machine and locate the screw!
Regarding adb, on my device I found the following in arc-setup-env:
# The IPV4 address of the container.
export ARC_CONTAINER_IPV4_ADDRESS=100.115.92.2/30
adb 100.115.92.2 (in Chrome OS's shell) works fine for me, the authorisation checkbox pops up and then good to go. su works fine through adb as expected. There's also a useful little nsenter script in Chrome OS to get into the android shell; /usr/sbin/android-sh, which I've been using in my script to help patch SE linux.
I actually just updated my rooting scripts recently to support 7.1.1, though I've only tested on my own Armv7 device (Flip C100).
I'll attach them to this post in case anyone wants to take a look. There's a readme in the zip, some more details can also be found here and below
EDIT: Fixed the SE Linux issue occurring with the previous version I uploaded (it was launching daemonsu from u:r:init:s0 instead of u:r:supersu:s0).
Anyone considering giving them a spin should bear in mind that the method does involve creating a fairly large file on the device as a rooted copy of the android rootfs. (1GB for arm, 1.4GB for Intel). There's a readme in the zip but the other couple of important points are that:
a) The SuperSU 2.82 SR1 zip also needs to be downloaded and extracted to ~/Downloads on the Chromebook.
b) Rootfs verification needs to be off. The command to force this is:
Code:
sudo /usr/share/vboot/bin/make_dev_ssd.sh --remove_rootfs_verification --force --partitions $(( $(rootdev -s | sed -r 's/.*(.)$/\1/') - 1))
or the regular command to do it is:
Code:
sudo /usr/share/vboot/bin/make_dev_ssd.sh --remove_rootfs_verification
c) If, subsequent to running the scripts, there's a problem loading Android apps (e.g. after a powerwash or failed install), the command to restore the original rootfs image is:
Code:
sudo mv /opt/google/containers/android/system.raw.img.bk /opt/google/containers/android/system.raw.img
Hey this is a great response.. thanks!
Nolirum said:
Hey,
There's no real documentation AFAIK, the thing is that ARC++ is a bit of a moving target, as it's so actively being developed/reworked. For instance, with the method described earlier in the thread - it started off being possible to just swap out a file location in arc-ureadahead.conf, then they changed it to arc-setup-conf, and now, since a few CrOS versions ago, the rootfs squashfs image is mounted in a loop fashion via the /usr/sbin/arc-setup binary instead, making an overview of the setup somewhat opaque to the casual observer.
Click to expand...
Click to collapse
verity
Yeah playing with it now, I'm looking at these /etc/init/arc-*-conf files... I see that the /dev/loop# files are being set up... (more below)
Nolirum said:
I was kind of hoping to implement a kind of hybrid systemless root style setup myself, but unfortunately I haven't really managed to find the time to sit down and fully figure out a few parts of the puzzle, in particular relating to minijail and working with namespaces. So, I'm still using the method mentioned in posts above for my rooting needs at the moment, the only significant changes being that at the moment I'm replacing /opt/google/containers.android.system.raw.img with a symlink to my writeable rooted rootfs img, and also that in recent CrOS versions the mount-as-read only and debuggable flags can be found in /etc/init/arc-setup-env ("Environment variables for /usr/sbin/arc-setup").
Click to expand...
Click to collapse
Sorry not sure what you mean by "hybrid systemless root style setup"? I take it you're modifying the startup script and replaced the squashfs file in /opt... my concern about doing it was whether they were implementing some kind of dm-verity equivalent to the squashfs file to make sure it hasn't been tampered with (say, by adding /sbin/su or whatever) or whether it's safe to replace that file.. Sounds like you're saying it is? (update: I guess that's what rootfs verification does, and we can turn it off....)
Also you mean arc-setup.conf:
env ANDROID_DEBUGGABLE = 0
right?
Nolirum said:
In general though, one can kind of get an idea of what's going on in the default setup by reading through the various /etc/init/arc-* Chrome OS upstart jobs (and their logs in /var/log). Though, like I say, things keep changing around somewhat with every CrOS update, as the implementation 'improves'. As time goes by, and the subsystem matures, it'll certainly be interesting to see what other approaches are possible relating to customizing Android on Chrome OS.
Click to expand...
Click to collapse
I hadn't realized the boot was still in flux-- I'd have figured they'd worked that out by now...
Nolirum said:
There should definitely be a write protect screw somewhere on the motherboard for the Samsungs, but so far I haven't come across any pics showing exactly which screw it is. So far, no-one seems to have been brave/foolhardy enough to fully tear down their own machine and locate the screw!
Click to expand...
Click to collapse
Heh.. not gonna be me..
Nolirum said:
Regarding adb, on my device I found the following in arc-setup-env:
# The IPV4 address of the container.
export ARC_CONTAINER_IPV4_ADDRESS=100.115.92.2/30
adb 100.115.92.2 (in Chrome OS's shell) works fine for me, the authorisation checkbox pops up and then good to go. su works fine through adb as expected. There's also a useful little nsenter script in Chrome OS to get into the android shell; /usr/sbin/android-sh, which I've been using in my script to help patch SE linux.
Click to expand...
Click to collapse
Cool-- adb connect 100.115.92.2 does indeed work I was gonna use netcat to open port 5555 in chromeos and pipe it through, but looks like nc isn't here and I'm not yet ready to start changing the FS..though probably will be soon... btw any idea which partitions get overwritten when chrome it does it's updates? Will /root and /etc get overwritten, for example... would a "powerwash" overwrite it or can you get easily get into an unbootable state on these things?
It's also kind of strange that adb is listening to port 30 at that (internal?) bridge address by default witho no UI to turn it off.. and it's inaccessible from outside.. i wonder if there's an easy way to change the bridge to share the same IP as the actual interface...
Final thought-- I'd love to build that system image myself soup-to-nuts, but I can't find any "caroline" device tree set up... do you or anyone else happen to know if there's a standalone AOSP device tree for the chromebooks? It would be cool to have a mashup AOSP/lineageos if such a think could be possible-- I'm guessing chromiumos is just taking the android tree, building it and then adding it into their build... I Haven't build chromiumos for many years now so I can't even begin to imagine how this android build integrates with the whole emerge thing they had going.. but I bet it takes a while
Nolirum said:
I actually just updated my rooting scripts recently to support 7.1.1, though I've only tested on my own Armv7 device (Flip C100).
Click to expand...
Click to collapse
Cool I'll take a look at these scripts.
So I haven't yet run the scripts-- just looking through them-- I noticed the section starting:
if [ -e /etc/init/arc-setup-env ]; then
echo "Copying /etc/init/arc-setup-env to /usr/local/Backup"
This doesn't exist on the x86 CB Pro. There's an arc-setup.conf that sets up the environment variables though. It sets WRITABLE_MOUNT to 0, but then so does arc-system-mount.conf
Not sure if these are different between x86 and ARM or if it's just in the latest update.. but figured I'd let you know. Wanna throw thse scripts up on github somewhere? (Or I can do it) and we can maybe look at keeping them up to date and/or standardizing them? It wouldn't be hard to determine if it's running on ARM or x86_64 (uname -i for example)..
fattire said:
So I haven't yet run the scripts-- just looking through them-- I noticed the section starting:
if [ -e /etc/init/arc-setup-env ]; then
echo "Copying /etc/init/arc-setup-env to /usr/local/Backup"
This doesn't exist on the x86 CB Pro. There's an arc-setup.conf that sets up the environment variables though. It sets WRITABLE_MOUNT to 0, but then so does arc-system-mount.conf
Not sure if these are different between x86 and ARM or if it's just in the latest update.. but figured I'd let you know. Wanna throw thse scripts up on github somewhere? (Or I can do it) and we can maybe look at keeping them up to date and/or standardizing them? It wouldn't be hard to determine if it's running on ARM or x86_64 (uname -i for example)..
Click to expand...
Click to collapse
Oh, the arc-setup-env thing is intentional. There does appear to be another issue with the x86 version though. I've written up a detailed response to your previous post; it's in a text file at the moment so I'll copy it over and format it for posting here with quotes etc now - should only take a few minutes. Yeah, sticking them on github might be a good idea; I've been meaning to create an account over there anyway.
Yeah, so... Regarding the scripts, since I've put them up here for people to download - I should mention that the first person to test them (aside from me) has reported that something's not working right (I'm waiting for confirmation but I think he tried out the x86 version). It's likely either an error on my part when copying across from my Arm version, or perhaps something not working right with conditionals, meant to deal with the various OS versions ('if; then' statements, I mean). Once I find out more, I'll edit my earlier post...
fattire said:
Sorry not sure what you mean by "hybrid systemless root style setup"? I take it you're modifying the startup script and replaced the squashfs file in /opt... my concern about doing it was whether they were implementing some kind of dm-verity equivalent to the squashfs file to make sure it hasn't been tampered with (say, by adding /sbin/su or whatever) or whether it's safe to replace that file.. Sounds like you're saying it is?
Click to expand...
Click to collapse
Oh, sorry for being a bit vague - I just mean perhaps implementing a kind of systemless root à la Magisk/SuperSU (from what I understand of how these work) - avoiding the need to actually replace files in /system. Since I'm mainly just using su for the privileges rather than actually wanting to write to /system, I had the idea that perhaps a sort of overlay on e.g. xbin and a few other locations, rather than actually rebuilding the whole of /system, might be an interesting approach....
Yep, I've been replacing /opt/google/containers/android/system.raw.img with a symlink to my modified image lately. Works fine... I think they've been focused on just getting the apps working properly, maybe something like dm-verity is still to come.
Although, one of the cool things with Chromebooks IMO is that once the Developer Mode (virtual) switch has been flipped, the system's pretty open to being hacked around with. I think a large part of the much-trumpeted "security" of the system is thanks to the regular mode/Dev mode feature, once in Dev Mode with verified boot disabled on the rootfs, we can pretty much do what we want (I like the message that comes up in the shell when entering the first command I posted under the spoiler - it literally says "YOU ARE ON YOUR OWN!").
So yeah, with Dev Mode switched off, verified boot switched on, we can't even get into the shell (just the walled-off 'crosh' prompt), making the system indeed rather secure (but, for some of us, rather limited).
fattire said:
Also you mean arc-setup.conf:
env ANDROID_DEBUGGABLE = 0
right?
Click to expand...
Click to collapse
That's what I mean by a moving target, lol. On my device the Canary channel is at Chrome OS version 61; I think they started to move out some ARC++ (the acronym stands for Android Runtime on Chrome, version 2, if anyone's wondering, btw) environment variables to a separate file in version 60, or maybe 61. Problems with being on the more 'bleeding edge' channels include:
#Sometimes stuff gets broken as they commit experimental changes.
#Any updates sometimes overwrite rootfs customizations; the higher the channel - the more frequent the updates occur.
#Some of the stuff that gets updated, may later get reverted.
And so on...
fattire said:
I hadn't realized the boot was still in flux-- I'd have figured they'd worked that out by now...
Click to expand...
Click to collapse
Yeah you'd think so. Honestly, the more I use CrOS the more it seems like a (very polished) work-in-progress to me. Though, I guess most modern OSs are also works-in-progress though. (I don't mean the former statement in a critical way; I'm very happy that new features keep getting added to the OS - Android app support being a perfect case in point, that was a lovely surprise, greatly extending the functionality of my Chromebook).
fattire said:
Cool-- adb connect 100.115.92.2 does indeed work I was gonna use netcat to open port 5555 in chromeos and pipe it through, but looks like nc isn't here and I'm not yet ready to start changing the FS..though probably will be soon...
Click to expand...
Click to collapse
Netcat's not there but socat, which I haven't any experience with but have seen described as a "more advanced version of netcat", is listed in /etc/portage/make.profile/package.installable, meaning that adding it to CrOS is supported, and as simple as:
Code:
sudo su -
dev_install #(sets up portage in /usr/local)
emerge socat
I tried socat out and it seems to work, might be interesting to play around with.
fattire said:
btw any idea which partitions get overwritten when chrome it does it's updates? Will /root and /etc get overwritten, for example...
Click to expand...
Click to collapse
Theres a question. I forget some of the exact details now (gleaned from browsing the developer mailing lists and the documentation on chromium.org), but from what I do remember and my experiences tinkering, I can say:
The auto-update model uses kernel/rootfs pairs, e.g. at the moment my device is booting from partition 2 (KERN-A) with the rootfs being partition 3 (ROOTFS-B). My understanding is that with the next OS update pushed to my device, CrOS will download the deltas of the files to be changed, and apply the changes to partitions 4 and 5 (KERN-B and ROOTS-B), setting new kernel GPT flags (priority=, tries=, successful=), which will, post-reboot, let the BIOS know that 4 and 5 will form the new working kernel/rootfs pair. Then the following update will do the same, but with partitions 2 and 3, and so on and so forth, alternating pairs each time. It's a pretty nifty system, and I think something similar might be happening with new Android devices from version O onward (?).
So partitions 2,3,4,5 are fair game for being overwritten (from the perspective of the CrOS updater program). Partition 1, the 'stateful partition') is a bit special, in addition to a big old encrypted file containing all of the userdata (/home/chronos/ dir?), it also has some extra dirs which get overlaid on the rootfs at boot. If you have a look in /mnt/stateful/, there should also be a dir called 'dev_image', which (on a device in Dev mode) gets mounted up over /usr/local/ at boot. As I mentioned above, if you do
Code:
sudo su -
dev_install
you can then emerge anything listed in /etc/portage/make.profile/package.installable (not a great deal of stuff admittedly, compared to Gentoo), which gets installed to subdirs in /usr/local/. So I think stuff in partition 1; /mnt/stateful/, should be safe from being overwritten with an OS update. I think crouton chroots get put there by default.
Most of the other partitions don't really get used, and shouldn't get touched by the updater, here's a design doc on the disk format, and here's a Reddit post (from a Google/Chromium employee) mentioning dual booting from partitions 6 and 7.
fattire said:
would a "powerwash" overwrite it or can you get easily get into an unbootable state on these things?
Click to expand...
Click to collapse
It's not too hard to mess up the system and get it into an unbootable state, lol. The "powerwash" just seems to remove user data, mainly. If you change up (the contents of) some files in /etc, or /opt, for example, then powerwash, normally they won't get restored to their original state (unless you also change release channel).
But, as long as the write-protect screw's not been removed and the original BIOS overwritten, it's always possible to make a recovery USB in Chrome's Recovery Utility on another device, and then restore the entire disk image fresh (this does overwrite all partitions). Another thing that I did was make a usb to boot into Kali; I was experimenting with the cgpt flags on my internal drive and got it into an unbootable state, but was still able to boot into Kali with Ctrl+U, and restore the flags manually from there. (To successfully boot from USB, it was essential to have previously run the enable_dev_usb_boot or crossystem dev_boot_usb=1 command in CrOS). I understand also that the BIOS type varies with device release date and CPU architecture, and that Intel devices may have some extra potential BIOS options ('legacy boot').
fattire said:
It's also kind of strange that adb is listening to port 30 at that (internal?) bridge address by default with no UI to turn it off.. and it's inaccessible from outside.. i wonder if there's an easy way to change the bridge to share the same IP as the actual interface...
Click to expand...
Click to collapse
I think I saw something related to this on the bug tracker. If I come across any info, I'll let you know...
fattire said:
Final thought-- I'd love to build that system image myself soup-to-nuts, but I can't find any "caroline" device tree set up... do you or anyone else happen to know if there's a standalone AOSP device tree for the chromebooks? It would be cool to have a mashup AOSP/lineageos if such a think could be possible-- I'm guessing chromiumos is just taking the android tree, building it and then adding it into their build... I Haven't build chromiumos for many years now so I can't even begin to imagine how this android build integrates with the whole emerge thing they had going.. but I bet it takes a while
Click to expand...
Click to collapse
Yeah, I haven't built Chromium OS or anything, but apparently, there's an option to create a 'private' overlay for the build, which doesn't get synced with the public stuff.
I think that the higher-ups at Google might be still umming and ahing as to whether or not to make source code available for the Android container, it's certainly not been made public yet. Actually, I remember seeing a Reddit post from a Google/Chromium employee mentioning this.
"That article is a little misleading in terms of open source. While the wayland-server and services that communicate with the ARC++ container are open source, the actual ARC++ container is not."
Perhaps they're waiting to see how similar implementations of Android within a larger Linux setup (e.g. Anbox) fare.
There doesn't seem to be too much that differs from AOSP in the ARC++ container - a few binaries and bits and pieces linking the hardware to the container (e.g. the camera etc), maybe some stuff related to running in a container with the graphics being piped out to Wayland?, and so on.
Oh, I was searching the bug tracker for something else, and just saw this (quoted below). Looks like it might be possible to run AOSP based images on CrOS soon!
arc: Implement android settings link for AOSP image
Reported by [email protected], Today (72 minutes ago)
Status: Started
Pri: 1
Type: Bug
M-60
When ARC started without the Play Store support there is no way for user to activate Android settings. We need implement corresponded section that has
Title: Android settings:
Link: Manage android preferences:
Inner bug: b/62945384
Click to expand...
Click to collapse
Great response! I read it once and I'll read it again in more detail then will probably have questions For whatever it may be worth, my only experience with chromiumos was building the whole thing maybe 4 years ago for my original 2011 Samsung "snow" Chromebook-- and making a bootable USB (or was it an SDcard?) to run it on (with a modified firmware that did... something I can't remember.. i think it was basically a stripped down uboot and I remember adding a simple menu or something-- I think I was trying to bypass that white startupscreen or something..). However, after doing this a few times to play with it, I realized that Chromiumos without the Chrome goodies kinda sucks and I promptly forgot everything and went back to stock.
I did have it re-partitioned to run linux as a dual boot from the SD slot or something-- I remember using that cgpt thing to select the different boot modes and vaguely recall the way it would A/B the updates (which "O" is now doing)... but anyhoo I was using the armhf ubuntu releases with the native kernel and ran into all kinds of sound issues and framebuffer only was a little crappy so...
I'm gonna re-read in more detail soon and I'm sure I'll have questions-- one of which will be-- assuming that most stuff is the same on x86 vs arm, why are there two scripts? How do they differ?
ol. On my device the Canary channel is at Chrome OS version 61; I think they started to move out some ARC++ (the acronym stands for Android Runtime on Chrome, version 2, if anyone's wondering, btw) environment variables to a separate file in version 60, or maybe 61.
Click to expand...
Click to collapse
This is the -env file I'm missing, I presume?
I think that the higher-ups at Google might be still umming and ahing as to whether or not to make source code available for the Android container, it's certainly not been made public yet. Actually, I remember seeing a Reddit post from a Google/Chromium employee mentioning this.
Click to expand...
Click to collapse
It looks from the response that the gapps portion might be what's in question-- just like ChromiumOS vs Chrome has all the proprietary bits taken out?
Here's what I'd ideally like to see:
* Rooted Android, with a toggle switch to hide su in settings a la lineage (requires a kernel patch something like this one) + settings changes from lineageos
* adb access from outside the device-- critical for quickly testing apks from android studio w/o a cable. Basically put the chromebook in a "device mode" where adb is passed through... I'm going to see if I can pipe adb through with socat as you suggest...
* what else... I dunno watch this space.
An update from a couple of guys that have tested out the scripts on Intel: It seems to be that while they are able to launch daemonsu manually (with daemonsu --auto-daemon), it apparently does not seem to be getting launched at boot.
I am waiting for some more information on this. Previously, for Marshmallow, the script was setting up the app_process hijack method in order to to launch daemonsu at boot; to support Nougat I changed it to instead create an .rc file with a service for daemonsu, and add a line to init.rc importing it. This works for me, and from what I can gather, it copied/created all files successfully on the testers devices, too, so I'm not sure at this point what the issue is there.
Edit: Fixed the issue. I updated my previous post with further details.
fattire said:
I realized that Chromiumos without the Chrome goodies kinda sucks and I promptly forgot everything and went back to stock.
Click to expand...
Click to collapse
lol yeah. True, that.
fattire said:
...assuming that most stuff is the same on x86 vs arm, why are there two scripts? How do they differ?
Click to expand...
Click to collapse
It's literally just two things that differ: the few lines where we copy the su binary over e.g.
/x86/su.pie → /system/xbin/su, daemonsu, sugote
vs
/armv7/su → /system/xbin/su, daemonsu, sugote
...and also the size of the created container. The x86 container is about 30 percent larger than the Arm one.
I had a little look at how to determine the CPU architecture programmatically on Chrome OS a while back, but couldn't seem to find a reliable way of doing this, at least not without maybe getting a bunch of people with different CrOS devices to run something like, as you mentioned, uname -i (which returns 'Rockchip' on my device, uname -m (which returns 'armv7'), or such similar, and collating the results. It was just easier to do separate versions for x86/arm, rather than introduce more conditionals (with potential for errors). I'm certainly not averse to adding a check for $ARCH, and thus standardizing the script, as long as it's reliable.
fattire said:
This is the -env file I'm missing, I presume?
Click to expand...
Click to collapse
Yep! It's just the same few envs as in the .confs, moved into a new file. I'm fairly confident that the script's conditionals deals with them OK.
fattire said:
It looks from the response that the gapps portion might be what's in question-- just like ChromiumOS vs Chrome has all the proprietary bits taken out?
Click to expand...
Click to collapse
Yeah, although the respondant there perhaps doesn't seem to realise that he's talking to a Google/Chromium dev, the way he responds. Not that that makes anything he says in his post is necessarily less valid, though.
fattire said:
Here's what I'd ideally like to see:
* Rooted Android, with a toggle switch to hide su in settings a la lineage (requires a kernel patch something like this one) + settings changes from lineageos
* adb access from outside the device-- critical for quickly testing apks from android studio w/o a cable. Basically put the chromebook in a "device mode" where adb is passed through... I'm going to see if I can pipe adb through with socat as you suggest...
Click to expand...
Click to collapse
Interesting... I agree, those would both be useful additions to the functionality of ARC++...
Quick question-- has Samsung provided the source for the GPL components (including the kernel, obviously)? I looked here but didn't see anything...? Previously the kernel was included along with the chromium source and there was like a kernel and kernel-next repository.. but this was like five years ago. I think the codename for the samsung chromebook pro is called caroline... let me quickly see if I can find a defconfig in the chromium source...
Back.. nothing here in the chromeos-4.4 branch. Nothing here either in the master branch. Maybe I'm looking in the wrong branches-- master is probably mainline kernel. Also the directories.. it took me five minutes to realize it wasn't going to be in arch/arm - force of habit I guess. I'll keep looking unless anyone knows. This "chromium-container-vm-x86" one seems to have dm_verity as an unused option. Ah, this is looking promising.
...and... here!
So it would seem that this would be built as part of the chromiumos build system, which seemed to be half gentoo five years ago building out of a chroot and was kind of a pain to set up... still, I'm guessing that since it's got that weird script to make the defconfig, what you could do is use google's chromiumos build script to make the kernel image (with whatever changes you want), then, assuming that it doesn't care if you replace the kernel, just throw it over the right Kernel A/B partition and see if it boots and starts up chromeos... it's weird cuz the kernel has to do double-duty for chromeos and android.. but I bet you can just replace it and it would work fine...
I had a cursory go at building a couple of kernel modules for my Flip C100 a while back - I didn't get too far though, lol. People do seem to have had success building their own kernels and running them with Chrome OS though, as with most things I suppose it's just how much time/effort you're willing to put in.
I think I used this and maybe this, from the crouton project to guide me.
From what I remember, I just got fed up of all the arcane errors/config choices. I remember that even though I'd imported my current device config from modprobe configs, there were then such an incredibly long string of hoops/config choices to have to go through one by one, to then be confronted with various errors (different every time ISTR) that I think I just thought "screw this". I think there were some other issue with the Ubuntu version I was using at the time as well. I know that sort of stuff's kind of par for the course with kernel compilation, but I was mainly only doing it so I could edit xpad in order to get my joypad working, in the end I found a different solution.
It shouldn't be too much hassle though, in theory I guess.... Oh, also, in order to get a freshly built kernel booting up with the CrOS rootfs, in addition to the gpt flags, I think you might have to sign it, too? (just with the devkeys & vbutil_kernel tool provided on the rootfs), some info here, and here.
From what I remember, the build system would do whatever key signing was necessary.... although I do now remember you're right there was some manual step when I was building the kernel, but I can't remember if that's because of MY changes or that was just part of the build process.
I I just dug out the old VM (Xubuntu) I was using to build and, well, let's just say I'll be doing a LOT of ubuntu updates before I can even realistically look at this. I do kinda recall setting up the environment was a huge pain so I'm going to see if I can just update the 5 year old source, target the pro and just build the kernel image and see what pops out the other end. At least I won't have to deal with the cross compiler, though I think it should hopefully take care of that itself.
Interesting to see that those crouton projects have emerged (no pun intended) so I'll check them out too while ubuntu updates itself
Thanks for the github links.. I'm going to go read that wiki.
Update: Looked at it-- funny they just stripped out the chromeos-specific parts they needed rather than emerge everything which is smart. My only question is now that Android is involved, there's that script I linked to earlier that seems to say "if you want Android support you'll need these bits too"-- wonder if the same config scripts apply, and if there are any other device tree considerations as well...
I may play a bit and see how smoothly it goes.. Unfortunately I don't have unlimited time either :/
Also, please do let me know if you put the scripts on github and I can send you pull requests if I come up with anything.
Update: Finally updated like 3 major versions of ubuntu... the "depot_tools" repo had its last commit in 2013, so I updated that. Wow, this is so much clearer than previous docs... it looks like something called gclient is used now, which I configured with:
gclient config --spec 'solutions = [
{
"url": "https://chromium.googlesource.com/chromium/src.git",
"managed": False,
"name": "src",
"deps_file": ".DEPS.git",
"custom_deps": {},
},
]
'
that let me do gclient sync --nohooks --no-history ...which i think is updating the ancient source. I probably should have just started over, but anyway... we'll see what happens.
Update again: After updating with this new gclinet tool, it appears that the old repo sync method is still required as described here. That hasn't changed after all, so now I'm going to go through this old method, which will probably completely overwhelm my storage as it's downloading with history.. but anyway, in case anyone is trying this-- looks like the whole chroot/repo sync thing may still be how it's done... the /src directory described above may only be for building just the browser, not the whole OS...
...and here it is. I will have zero room to actually build anything tho, but hey.
* [new branch] release-R58-9334.B-caroline-chromeos-3.18 -> cros/release-R58-9334.B-caroline-chromeos-3.18
Note to self: use cros_sdk --enter to actually get in the chroot. Then:
~/trunk/src/scripts $ ./setup_board --board=caroline
to set up the build for caroline. Then to build:
./build_packages --board=caroline --nowithdebug
Useful links:
* Building ChromiumOS
* [URL="http://www.chromium.org/chromium-os/how-tos-and-troubleshooting/portage-build-faq"]eBuild FAQ
[/URL]