Related
>>>> 24Jan2012_2155 - linboothkvc v1.1 source - does a bit more thorough job with Cache flushing for the corner cases where the new guy in the Nirvana environment doesn't do a thorough job of cache invalidation. Now have added the source to this post itself (cas, it is pretty much done now, except for any forgotten corner cases, and also one pass at removing all dependence on hidden kernel functions which I shamelessly depend on - rather it is only setup_mm and cache flush walking, others (on 2nd thought rather all) are trivial to replace but have left it there just from a future proofing perspective). However the details or blah blahs are in the newer posts towards the end of the thread. <<<<
>>>> 22Jan2012_1430 - linboothkvc v1.0 source with working binary kernel module for Nook Tablet released - look towards end (may be page 2) for the source - As you would already know, it can also work with any rooted arm based linux device provided it is recompiled for the given linux kernel on that device along with updated kernel function addresses in lbhkvc_k.c and appropriate Nirvana (some minimal change, if required) and NChild code (ie the bootloader you love for your device) <<<<
>>>> 22Jan2012_0058 - I have uploaded the source for a working linboothkvc for Omap3/Omap4/Arm_Cortex_A SOC based devices. As far as linboothkvc is concerned, it works on NookTab also successfully. However there is some effort/love still required wrt the NChild or bootloader used from with in the Nirvana environment ;-) <<<<
>>>> 17Jan2012_2325 - FINALLY SUCCESS on NookTab also ALSO Note that the POWER of KERNEL Modules and linboothkvc in turn is well beyond NookTab for the adventurers ... ;-) <<<<
>>>> 16Jan2012_2326 - Beta version of code posted towards the end, Now it fully runs on BeagleXM Hw - with minimal love should run on NookTab also, so enjoy <<<<
>>>> 15Jan2012_0251 - Alpha version of source code in post towards end, fully runs in Qemu for now ;-). And the difference between this cup and the lip (i.e actual h/w) being the missing proper cache handling from my side <<<<
Hi All,
Before my ideas with init hijack, and uboot hijack, I had a idea with trying to implement a kernel module to allow execution of any code, after killing linux without a full reboot (which would give control back to secure boot). However 2ndihkvc and NOPBypass came in between this.
However I will try and put sometime into it, as and when possible. I have some work coming up over the next few days, so going will be bit slow compared to my other two threads, but if nothing turns up from BN side (what I heard from Adamoutler on the other thread) wrt open ended bootloader then I will spend bit time on this.
Note that one doesn't require kexec to achieve functionality similar to kexec ;-). Linux kernel modules is a very powerfull mechanism which we have in our hand to give lot of or all the control required (Unless linux has changed drastically over the last few years, when I have been away from it, but that seems less likely, even thou I have heard and discussed some ideas which curtail this power almost a decade back, but I don't think it has materialised yet, which should be good for the situation we are in, and hope it remains like that for the forseeable future, what with all these close minded companies and closed devices these days).
NOTE: Look to the newer posts below for the Source Codes. RC1 source code in a day or two (but has mentioned, no significant changes wrt Beta
[REPOST] MAYBE Exec_Anycode instead of kexec
>>>> This was my old post on bypass bootloader ideas thread, put here for completeness <<<<
*** MAY BE A POSSIBLE EXEC_ANYCODE logic instead of KEXEC***
NOTE: KEXEC tries to run another linux kernel, so may be its logic is more complicated, than if we are trying to run just any code in kernel or better privilage level. I haven't looked into kexec as of now, so I am only guessing about kexec complexity, beyond what I have mentioned below for my method (which again I haven't tried as of now, just a idea).
If one is trying to run something from memory when already in Linux, then one also has to worry about the privilage level at which the processor is running, as well as about page table mapping etc... If the other thing which we are trying to run is another kernel or x-loader or uboot or another bootloader for that matter.
So if kexec doesn't work, may be there is another option available which is to
a) Create a kernel module (NOTE: It runs with same privilages as the kernel) which does
a.1) Disable interrupts so that control doesn't get out of our code
a.2) Over write the reset vector with x-loader or what ever custom bootloader one wants to get control of.
a.3) Change the memory map to have a 1-to-1 map for the region where the code is currently running (or rather for the code which will be run in the next step) or overwrite a region which already has 1-to-1 map between Physical and Virtual addresses, with the code for the next steps. Go to next step.
a.4) Disable page tables (I haven't tried this before, ie after it has already been enabled, but I don't see any reason why ARM doesn't allow this - except for things like what we are trying).
a.5) Change to the reset ARM privilage level (If a soft reset doesn't do it already, haven't looked at ARM at this level for ages, so don't remember)
a.6) Trigger a soft reset
This should give control to the bootloader which we have loaded into the reset vector address. (Rather we should trap all possible exceptions i.e all the 8 or 16 or what ever is the number of exception addresses in the exception vector).
NOTE: If we are not able to change back to the reset time privilage level for the ARM processor, then we should be still able to have a modified Linux kernel, which doesn't try to switch ARM privilages if not required - this I have tried, ages ago, If I am not wrong as part of some other activity I had done.
Current thoughts
Hi,
On thinking once again
a) We definately want to disable interrupts, so that we don't lose control other than for exceptions if at all.
b) We may have to stop the other processor if it is up in SMP, but I have noticed that most of the time, the other processor is shutdown in NookTab.(Have to think thro this bit more, later).
c) May be trap the TLB exception handler and inject custom pagetable till MMU is switched off (An idea for now, have to experiment a bit later). Rather than going thro the linux mmu code (am getting bit old, a decade back I would have done the required linux magic in few jiffies, but have been away from linux for too long now . OR force few entries into current/kernel linux memory map.
d) Copy
d.1) the core code required to manage stuff to a 1:1 mapped region.
d.2) Also the code required to jump to like (i.e the bootloader or what ever)
Other d.2) Or implement minimal code (as part of d.1) to read uart (oops - now that would have cut the complexity by 3/4th, but for people hating uart these days;-) or may be sd card and get a sector into memory (like x-loader).
e) Disable the mmu
f) Jump to the required code location of new universe
Today I did a quick running/jumping glance thro kexec code once, and even it does some what similar only, if I am not wrong, except for may be it not hijacking the TLB exception handler or having some junk for debugging or so ..., so I am not that off wrt the required idea.
First baby step - try and understand the default memory map
Hi,
Did try the 1st baby step towards this, by trying to go thro the running systems memory map.
LBHKVC driver v04Jan2012_2110
INFO: Total memory is iTotalMem 0x40000000
INFO: Page Size ??? PAGE_SHIFT 12 ie 4096
INFO: Begining of Platform ram PHYS_OFFSET 0x80000000(0x40000000)
INFO: End of userspace mapping TASK_SIZE 0xbf000000(0x7f000000)
INFO: Start address for Modules Space MODULES_VADDR 0xbf000000(0x7f000000)
INFO: End address for Modules Space MODULES_END 0xbfe00000(0x7fe00000)
INFO: Permanent kernal mappings PKMAP_BASE 0xbfe00000(0x7fe00000)
INFO: Kernel direct 1:1 map ??? platform RamBeg PAGE_OFFSET 0xc0000000(0x80000000)
INFO: Kernel direct 1:1 map ??? platform RamEnd high_memory 0xf0000000(0xb0000000)
INFO: vmalloc/ioremap space Begin VMALLOC_START 0xf0800000(0xb0800000)
INFO: vmalloc/ioremap space End VMALLOC_END 0xf8000000(0xb8000000)
Now one thing which is potentially true and easy is, the Physical Memory from 0x8000.0000 is 1:1 mapped to 0xc000.0000 in a linear way (Have to validate, but should be). There are certain virtual to physical maps which seem bit odd, most probably I am using the wrong function to do the conversion and or traversal of pagetable. Also some of them are not necessarily meant to have a mapping other than act has markers conceptually (Have to verify later).
Having something same to same mapped would have kept things to a minimal, and made this too trivial now - why not ooh linux gods ;-(. Now this is forcing some hijacking or forcing of page table entries or experimenting and seeing if this is good enough to me/any one else interested.
Also there seems to be something called SAR_RAM, have to see if this is of any use. As well as some of the regions reserved thro kernel command line and see if anyone is using it, or if it can be evicted if required, there are only for funny experimentations.
Otherwise I think, I should be able to get even my current modules virtual address translated to physical address and may be same-to-same mapped if required. Or forcefully take some region beyond my current running code and any exception logic I require.
Also I have to check how critical is the same-to-same map if any when switching off mmu, or is 1:1 map good enough, have been away from lowlevel arm also for too long now.
Wow hkvc you respond quicker then I do.
First off it is not a 1 to 1 mapping if you look at the arm B&Ndefconfig youll see that they utilize a different way of mapping (not looking at the code right now).
Second off kexec with the kernel module that does KEXEC_LOADED is essentially this. I would look at the kexec.c code, and you will see that you can comment out the sanity checks in the find hole function and it will find a valid hole, then with injection you could make this work. However, while i have done embedded design and some hacking, kernel modules are not my specialty. We need a high priority kernel module that removes interrupts, so that the kexec code can load.
baby step2 of linboothkvc
Hi Loglud/All,
Thanks for your inputs.
Not sure about ur (Loglud) 1:1 map part comment. Currenty when people say 1:1 I am not sure whether they mean linear map (i.e with a constant addition or substration you can get virtual - to - physical mapping and inturn the other way round) or same to same mapping, I have to look at arm initial booting code and see what they mean (and in turn what ARM requires), because they require this when booting and inturn the mmu gets enabled.
Also the code to dump the map what I put above, was my first attempt at kernel level view of memory maps after ages, so there are some FUNNY ;-) errors in the way I tried to dump it quick and dirty.
If you are talking about 0xC000.0000 (Virt) to 0x8000.0000 (Phys) mapping which I mentioned, I still feel it is linear mapping, but I have to verify once. Linux kernel used to maintain a linear mapped region to simplify the internal management of physical pages(i.e memory), so that they can convert from virt2phy and back easily wrt physical memory with out requiring to go thro pagetables and for other reasons (Obviously with limits, i.e the Full memory is not linearly mapped but the initial 800MB or 900MB or so used to be). I will verify it later.
Either way independent of that I have potentially found the region which I will be attacking wrt getting the required same-to-same mapping (which I want to use, even if linear map is good enough, which again I am not sure is true at this time). The VIRTUAL ADDRESS space set aside for kernel Modules in Linux (MODULES_VADDR (0xbf00.0000 ...) and MODULES_END) overlaps with the actual physical memory location (0x8000.0000 - 0xc000.0000) here. And there is enough free memory in this module virtual address space, as there are only 2 modules loaded in NookTab, plus the region is originally there to allow lot of modules plus worst case I don't really worry much about stomping_on/reusing someone elses used memory because I am going to kill the system shortly and I have already locked the current cpu by blocking interrupts.
Inturn I will initially see if there is a physical page in the physical address space from 0xbf00.0000 to MODULES_END such that I can do a direct same-to-same mapping into the corresponding Virtual address space. Again even if there is non, at one level I may not have to mind really ;-) as I will be nuking the full system shortly.
I am looking at same-to-same mapping to allow the code which will do the mmu disable to continue to work before and after mmu disabling. While a Linear mapped region is good to load the code (bootloader, kernel, ....), before mmu is switched off, which will be passed control after mmu has been disabled, and which inturn can welcome the new system.
NOTE: I am in this more for the fun of exploring, so at least initially I don't want to use kexec and modify it (If I fail, then I may look at using it directly, but don't see a reason to fail for now), rather I want to come up with a concept (and as you rightly mentioned, even kexec follows (and it definitely should) similar concept to a great extent except for may be some of the implementation steps, as end goals are similar) and then implement it for the fun of it, even in crazy ways if possible (I am just starting out on this now, so I dont want to say one way or the other on this aspect now) just for it
NOTE: I am explaining my thoughts, so if someone else is interested in experimenting parallely on his/her own, they can get some ideas (good or bad
BabyStep3 basic implementation done - but results say long fight ahead
Hi All,
Over yesterday and today, I have implemented a basic logic to test minimal kexec equivalent logic using kernel module.
Rather had to dig thro the kernel source code to
a) refresh thro my kernel basics and to understand atleast some of the changes in the newer kernel versions and MORE importantly
b) to OVER COME the tendency of core kernel developers to DISABLE EXPORT of some of the useful functions to external kernel modules.
Eitherway most of the disabled symbols I could pickup and hardcode the address in my code to still access them indirectly using function pointers - what would world be with out function pointers or rather pointers in general.
Also because of the SMP nature of the SOC, had to dig thro some of those stuffs also. Then for now decided to use some of the support mechanism already available within kernel to help with kexec logic like fin, reset, etc to try and see if I could use the easy path into it for now initially
However what I seem to have realised/found is that
a) Either these support routines haven't been fully implemented in the used kernel version on NookTab currently and or
b) I am missing few more additional steps wrt SMP (Have already killed the 2nd Processor using proper api in kernel - Have to cross check the CP15 to verify for sure once again later).
It seems to be mostly (a) and inturn related to cache cleanup and may be mmu switching, have to debug further.
Otherwise the same logic seems to be working in BeagleboardXM (rather within qemu -M beaglexm) except for some reason the uart messages seem to disappear once I switch over even thou the code seems to be running in the new Same-to-Same map with the physical memory address part, with proper UART address - checked using info registers in qemu (Have to debug this part of qemu related to how it decides which uart to show in Ctl-Alt-3 etc bit more and or try on a physical board sometime next week).
In few days time I will upload the code I have come up with (even thou useless from using/achieving kexec logic perspective currently, still may provide some ideas or act as a base platform for someone wanting to experiment, but having initial inertia , if I am not able to spend more time on it.
Initial Pre-Alpha source for linboothkvc kernel module and utils
Hi All,
As promised, I am uploading the initial version of the kernel module source code (with lot more updates compared to last weekend, when I mentioned about it) for achieving kexec like functionality even if kexec is disabled in kernel. As I always told, kernel modules are equivalent to kernel, so you can do what ever you want in kernel module that can be done in kernel, provided one is bit patient ;-).
Note: This is pre - alpha version of code for people with initial inertia/starting trouble to experiment. This has a known bug with cache handling which I have to fix as well as some restructuring and cleanup to do.
This will currently only run in Qemu, because it doesn't bother much with Cache. But on ACTUAL targets it will fail Randomly (there is a very very small one in a million chance that it can succeed if all the stars line up and the cache gods support you
This is no longer as critical for NookTab has it was 1 week back, because Now some people have already released a exploit which uses a uboot bug ;-(, which ideally we could have with held for a future product (because NookTab was already sufficiently open and didn't need any more exploits to use its full potential, as I had mentioned last week). But either way I realise that people are very impatient these days. And congrats to those people for their work however.
For people who want to experiment, the initial skeleton is available in this release
Alpha release with embedded x-loader for BeagleXM Qemu
Hi,
NOTE: This release successfully bootstraps a new linux from with in linux in Qemu, my yesterdays release would have also done the same, but would have required additional code to handle the final nitty grities, this is taken care of now in this release. So that it is easier for people wanting to experiment.
I have updated the kernel module to allow two images to be embedded into it.
a) The initial boot strap loader called Nirvana (Examples like bloop0.S, bloop1.S, ... and now a full fledged omap3callbootrom1.S for BeagleXM on Qemu - Rather it in itself is independent of Qemu-BeagleXm or Hw-BeagleXM)
and
b) The actual bootloader/???? to run in the new prestine environment called Nirvana's Child (NChild). Currenlty it is a version of the x-loader for BeagleXM (Either Qemu or Actual BeagleXM).
In turn once the kernel module is done with its job. It passes control to Nirvana. Also it passes the physical address and length of the NChild image to Nirvana thro r0 and r1.
The Nirvana code in turn can decide what it wants to do with the system. The default Nirvana code i.e omap3callbootrom1.S takes care of copying the NChild(by default x-load.bin) to 0x4020.0800 and then pass control to 0x4001.4000 so that it loads x-loader properly (ie setup stack for all modes etc).
NOTE: As of now it works on Qemu only. My earlier release yesterday would have just printed 1s on the screen, while this version actually boots into a new linux kernel + system in Qemu from with in a already running Linux system in qemu.
However it won't run outside Qemu successfully, as I haven't yet had time to look at fixing the cache issue, because I had to add support for NChild image logic either way for doing anything useful with the code.
NOTE: As this is either way no longer critical for NookTab, I will take a stab at it based on my hack-vs-life balance also. Depending on what and all come up over the next few days in life.
HOWEVER this latest release has the FULL REQUIRED INFRASTRUCTURE/LOGIC for running this on a actual h/w expect for the missing proper cache handling. It also includes a x-load.bin by default for BeagleXM which can be used either with Qemu-BeagleXm or Hw-BeagleXm.
Beta Source - Success on a Hardware (beaglexm for now) should work on NT also ...
Hi All,
I have finally identified the stupid cache issue which was frustrating me and eating my head for long and stalling this project unnecessarily . However it gave a nice oppurtunity for me to dig thro the kernel code as well as Arm documentations - which is what I am after either way So ALL IS WELL in the end
The problem was related to kernel's normal code using MVA based cache operations for flushing, which in the newer architectures stops mostly at L2 cache rather than hitting the memory. This is fine for Normal linux kernel operations because they don't disable cache and so the proper data will get used. But for us as we want to disable cache to give a true pristine NIRVANA environment, this doesn't work, as memory contains old/stale/wrong data. In our code flow This even affects the KERNEL MODULE CODE itself, leave alone Nirvana or NChild code.
Once I realised this by digging thro documents as well as seeing the strange behaviour (rather unbelieavble, initially for me) of my code(kernel module as well as Nirvana and NChild)/logic after cache disable, I was able to get it RUNNING SUCCESSFULLY on BeagleXM actual hardware and not the qemu emulation which I was previously using.
As of now it is restricted to Omap3/BeagleXM, because the Nirvana code Omap3callbootrom2 makes use of the bootrom memory map usage knowledge to setup NChild appropriately and pass control to it thro Bootrom (so that stack is setup for all modes). With small effort the same can be changed or updated for Omap4 and NT should be in the fold (Unless SMP creeps up, inspite of me shutting down the 2nd processor, for some crazy reason in the worst case ;-)
NOTE: Also my last release (alpha) for Qemu beagle had a bug which was a blessing in disguise, in that it was not disabling the MMU from with in kernel module(which I had added just for the heck of it and is actually not required). However it was still doing it once it hit Nirvana code, as required. If I had not done the mistake of using mrc instead of mcr with in my kernel module code, it would have failed immidiately, because BeagleXM has only 512MB ram and the kernel module code space is at virtual address 0xbf00.0000 or around it, which is WELL BEYOND the 512 MB of physical memory, so there would not have been any 1-to-1 memory map for it and disabling MMU would have made things go crazy. Note that Nirvana and NChild are kmalloced into linear mapped kernel address space which is with in physical memory limits normally ;-).
NOTE: May be there is a watchdog timer or coprocessor or so which I have to disable, haven't looked into this yet, which seems to mess things up, if I stay in Nirvana code for too long. However as by default there is no need to remain in Nirvana code for long, as it is required to pass control to NChild as quickly as possible, this is not a immidiate issue to worry about now.
Let the experimentations begin
For H/w BeagleXM boot, bypass stupid SMI based L2 Cache maintaince rot in uboot ;-)
Hi All,
If anyone has tried running my yesterdays Beta release on BeagleXM h/w there is one update and one other IMPORTANT things to keep in mind.
a) Update the omap3callbootrom2.S to directly call into NChild rather than thro BootRom, that call into BootRom is not required. i.e jump to 0x4020.0800 instead of 0x4001.4000 at the end.
NOTE: This also makes the code more generic and usable with minimal love across Beagle and NookTab.
b) In U-Boot remember to DISABLE/BYPASS the calls to ROM Support routines thro SMI to setup L2 cache as well as to invalidate cache. This is no longer required in newer Omap3 chips as well as there is actually few bugs in u-boot code itself related to this, as one wants to clear full cache but SMI rom routine only clears L2 (also the full cache walking for invalidate is there following the SMI call, so bypassing doesn't lose functionality), if I get rom code description in TRM correctly, plus few other bugs atleast in rowboat version. The files board.c and cache.S contain the calls to SMI which has to be bypassed.
I have attached the minimal patch required to uboot to allow it to work with linboothkvc with BeagleXM.
With the above two changes, linboothkvc will always succeed in BeagleXM, enjoy
NOTE: In my setup today, I have modified the NChild x-loader to load u-bootk.bin rather than u-boot.bin. Thus I can have both the Normal u-boot.bin for Normal booting and u-bootk.bin for linboothkvc based booting . My Beta release doesn't contain this modified x-loader NChild, the next release will contain the modified one. But with source of x-loader, you can do it yourself and copy over as NChild into linboothkvc
Success on nook tab
Hi All,
LinBootHKVC has successfully booted into Nirvana code in NookTab
NOTE: No major change required to my beta release other than one mentioned in my last post to make my released Nirvana code more generic and the address update in this post for NookTab.
As I had mentioned yesterday, even thou I hadn't got the time yesterday to check it on NookTab yet, it should work 90% except for any crazy SMP issues, inspite of me disabling the 2nd Processor. WELL IT turns out that what I had done already was sufficient for NookTab also.
Only in omap3callbootrom2.S you have to change the sram address to which things are copied from 0x4020.0800 to 0x4030.0800 for NChild code. However I haven't crosschecked the x-loader based NChild on NookTab as of now, but DONT SEE ANY REASON why it should fail, other than for any issues with stupid code, like the SMI calls to manage L2 cache in u-boot for beaglexm and for some reason if 0x4030.0000 space has some issues I haven't thought of in Omap4 (I am relatively new to Omap4 started with NookTab only).
Will upload the Release Candidate version 1 of the code in a day or two. It is late here and I have been up on this NookTab project for few weeks now
a) starting from idea of linboothkvc
b) then moving to 2ndihkvc
c) followed by NOPBypass with UART access
0) The uboot loop hole (Oh my my my (But not released from my side hoping to keep it for future, but alas, if only people have/had patience ..., that is partly a dream now
d) followed by MenuK for 2ndihkvc - haven't released yet, time got sucked back
into linboothkvc
and back to
e) linboothkvc
all of the above work on NookTab successfully as of today and all can be used to achieve custom Roms and in case of NOPBypass and linboothkvc even custom kernels
So enjoy everyone. Some sweet rest for me atlast ;-)
NOTE: The POWER of Kernel MODULE and inturn linboothkvc goes well beyond NookTab for the adventures people out there
Oh My MURPHY - For now keep away from 0x4030.xxxx
Hi All,
Now as Murphys law would have it , 0x4030.4350 (the default address used by x-loader in Omap4) has some issue with it (which I have to debug later). So if trying on NookTab remember to use some other address (what other I will leave as a exercise to the interested
Also the x-loader from BN and or inturn from Ti expects a special meta data structure to be passed along thro r0, which inturn tells it about boot device and boot mode. So you will require to take care of this or better still simplify the structure to handle in cpu/omap4/start.S
Also I did the cardinal sin of doing too many changes when working/debugging on a new unknown device (from my perspective i.e NookTab) with no jtag access (atleast at my end for now).
So ended up digging into Arm L2X0 cache and inturn PL310 trm and writing a L2X0 cache flush logic (obviously also cross checking the equivalent logic in Linux kernel) when none was required in reality as I had already found and mentioned in my last post few days back. I forgot (rather got too lazy to checkout my own old code, just for the fun of exploring further ;-) my own Nirvana code which I had tried towards the beginning of the week which had successfully run and printed stuff on screen.
Also dug into the Address translation support registers in CP15 to be 100% sure that the 1-to-1 map was in fact there (again lazy to dig thro the linux kernel setup_mm code, when arm walks the page table tree for you and on top when I had not tried this arm instruction method before .
But ended up finally realizing that me using the 0x4030.4350 was the culprit atleast for now.
So keep away from that address for now (or debug further on this on your own for now, I will be looking at it only later, could be something to do with HS code using that region or ..., there is more interesting stuff to do for now) and remember to patch x-loader appropriately and you should be able to get it running on NookTab; again with my last code release with hardly any changes other than what I have already mentioned in the last 2 to 3 posts.
A updated source with the now useless l2x0 cache flush and va2pa address translation verification logic and a newer nirvana code with panda board support also (verified works perfectly - only slight changes required between Panda and NookTab as far as Nirvana code is concerned) I will release after some more experimentation and code stabilisation at my end.
hkvc said:
Hi All,
Now as Murphys law would have it , 0x4030.4350 (the default address used by x-loader in Omap4) has some issue with it (which I have to debug later). So if trying on NookTab remember to use some other address (what other I will leave as a exercise to the interested
Also the x-loader from BN and or inturn from Ti expects a special meta data structure to be passed along thro r0, which inturn tells it about boot device and boot mode. So you will require to take care of this or better still simplify the structure to handle in cpu/omap4/start.S
Also I did the cardinal sin of doing too many changes when working/debugging on a new unknown device (from my perspective i.e NookTab) with no jtag access (atleast at my end for now).
So ended up digging into Arm L2X0 cache and inturn PL310 trm and writing a L2X0 cache flush logic (obviously also cross checking the equivalent logic in Linux kernel) when none was required in reality as I had already found and mentioned in my last post few days back. I forgot (rather got too lazy to checkout my own old code, just for the fun of exploring further ;-) my own Nirvana code which I had tried towards the beginning of the week which had successfully run and printed stuff on screen.
Also dug into the Address translation support registers in CP15 to be 100% sure that the 1-to-1 map was in fact there (again lazy to dig thro the linux kernel setup_mm code, when arm walks the page table tree for you and on top when I had not tried this arm instruction method before .
But ended up finally realizing that me using the 0x4030.4350 was the culprit atleast for now.
So keep away from that address for now (or debug further on this on your own for now, I will be looking at it only later, could be something to do with HS code using that region or ..., there is more interesting stuff to do for now) and remember to patch x-loader appropriately and you should be able to get it running on NookTab; again with my last code release with hardly any changes other than what I have already mentioned in the last 2 to 3 posts.
A updated source with the now useless l2x0 cache flush and va2pa address translation verification logic and a newer nirvana code with panda board support also (verified works perfectly - only slight changes required between Panda and NookTab as far as Nirvana code is concerned) I will release after some more experimentation and code stabilisation at my end.
Click to expand...
Click to collapse
hkvc:
Thanks for all of your work. Even though the bootloader bypass has been found, you do not know how many others you are helping. This could be used as an indefinite alternative to locked bootloaders for ALL DEVICES! Keep chugging on this, and I'm sure you'll find your solution to the l2 cache flush. Also i have been looking around, and I was curious if this could be another explotable boot flaw,
Code:
/*
* The SAR RAM is maintained during Device OFF mode.
* It is split into 4 banks with different privilege accesses
*
* ---------------------------------------------------------------------
* Access mode Bank Address Range
* ---------------------------------------------------------------------
* HS/GP : Public 1 0x4A32_6000 - 0x4A32_6FFF (4kB)
* HS/GP : Public, Secured
* if padconfaccdisable=1 2 0x4A32_7000 - 0x4A32_73FF (1kB)
* HS/EMU : Secured
* GP : Public 3 0x4A32_8000 - 0x4A32_87FF (2kB)
* HS/GP :
* Secure Priviledge,
* write once. 4 0x4A32_9000 - 0x4A32_93FF (1kB)
* ---------------------------------------------------------------------
* The SAR RAM save regiter layout is fixed since restore is done by hardware.
*/
27.4.4.4.1 Public Use of SAR RAM
At system level, the OMAP4430 SAR RAM memory is divided into four banks. The public ROM code uses only the first bank, which is always public-accessible. More specifically, the software booting configurationstructure must be located in the upper 1.5KB of the first bank.
The public ROM code offers some flexibility about the location of the software booting configuration structure. The PUBLIC_SW_BOOT_CFG_ADDR pointer defines the start address of the structure within the SAR RAM bank (see Table 27-14).
As mentioned previously, the software booting configuration feature is optional. Hence, the public ROM code decides to use the feature based on the value read on a warm reset at the address pointed to by the PUBLIC_SW_BOOT_CFG_ADDR pointer. If the value matches the range 0x4A326A00 – 0x4A326FFF, the ROM code tries to extract the structure located at that address. The value pointed to by PUBLIC_SW_BOOT_CFG_ADDR is always overwritten to zero on a cold reset.
The recommended address for storing the software booting configuration structure described hereafter is defined as PUBLIC_SAR_RAM_1_FREE. It is, however, possible to locate the structure at any location within the 1.5-KB range.
It is moreover possible to use the public SAR RAM area for any other purpose, such as storing traces for HLOS use. Obviously, care must be taken not to overwrite the locations used for low-power modes and/or software booting configuration if used.
Click to expand...
Click to collapse
linboothkvc v1.0_RC3 with good x-loader for O3Beagle and O4Pandabrd and semi 4NookTab
Hi All,
I am attaching the source code for linboothkvc with a good basic Nirvana code for Omap3 (Many mobiles and few tablets) and Omap4 (few mobiles and tablets) devices. NOTE: for NookTab, basic linboothkvc works perfectly now (except for may be some toning down of the secure Monitor code if possible), However there is some more work required at x-loader (which I am using as my NChild code) level which I have mentioned further below as TODO. Otherwise the basic logic fully works for NookTab also now. At this stage it can be used by developers and not end users as NChild code for NookTab still requires some love.
[A] obviously there would be some minor tweaks to the NChild load address and args to pass etc based on x-loader/bootloader used in a given device, but still the basic skeleton is fully there in Nirvana code now. And the same can be modified for other SOCs from Samsung/NVidia/(Not getting the other vendor name now funny)... pretty easily.
Also the addresses for the linux kernel functions which I use require to be updated for any new device or for a new/different kernel for the devices which I have already put the code in this release.
[C] It expects to find a u-bootk.bin in mmc vfat root dir, provided you are using the x-loader code which I have bundled. How ever based on your device, you may have to change to a different bootloader or modify x-loader to suit that devices need, and thus With your own version of x-loader or NChild code, you can always modify it the way you want as to what it loads and from where.
But with the above changes/setup this can be used with any Rooted ARM based Linux device (be it android or webos or ...)
Changes I had to do to x-loader for NookTab:
[1] I had to bypass or find alternate locations for some of the 0x.4030.xxxx addresses used in x-loader from BN/Ti (Haven't had time to look into the details as to why this is forced on us and how to force the use of the same address yet). For this reason I am recompiling the x-loader with a load address of 0x8000.7ff0 for now and sidestepping it mostly.
[2] Clock and related init settings related to MPU,IVA and DDR memory are offlimits for now (but then again for a basic working these need not be changed at x-loader level eitherway).
[3] smc/SEC_Ppa functions are offlimits for now, again not required at x-loader level.
[4] TODO: the x-loader bundled with BN source seems to be a old version or has some non required (from BN perspective, but usefull from our perspective) logics for BN stripped. So even thou I have modified the high level logic to do a FAT load from MMC for u-bootk.bin. Because of this missing support for FAT boot mode, the current version of x-loader for NookTab which I have bundled in my code doesn't load u-bootk.bin.
Enjoy and happy Experimenting all
NOTE: When I say bundled x-loader, it is only binary blob, you can get the actual source code for x-loader from the respective git sources. I think few days back I uploaded the patch required for beagle x-loader to make it run in linboothkvc Nirvana env. I will do the same for x-loader for NookTab and Omap4 (rather omap4/panda hardly any change required from Ti release - I had to mainly simplify the argument mechanism passed to x-loader and nothing much, if I remember correctly, as I tried pandaboard 1 day back and after that I have done so many other things that, pandaboard is out of my memory Fifo) in few days.
[DONE] source of linboothkvc v1.0 (includes binary for NookTab with working xloader)
Hi All,
LinBootHKVC for NookTab is fully DONE now. Well (for developers) it can work with any ROOTED ARM based LINUX device (android or not doesn't matter) with kernel module support with minimal love, for those who are interested ;-).
Along with the source (which can work with any Arm based linux setup with some minimal love), this release also includes the binary kernel module for BN NOOK TABLET with firmware 1.4.0. Which inturn contains a working x-loader binary as its NChild for NookTab, which looks for a u-bootk.bin file on the uSD card.
To use this on NookTab (similar steps work for any other device, provided you have suitably compiled linboothkvc for your device with proper love)
a) get your required/prefered u-boot.bin file and prepend a 288 (0x120) byte dummy header and name it u-bootk.bin
i.e
dd if=/dev/zero of=/tmp/dummy.bin bs=288 count=1
cat /tmp/dummy.bin u-boot.bin > u-bootk.bin
b) copy this u-bootk.bin to the root directory of the 1st partiton on a uSD card which inturn should be VFAT formated. (To be 100% safe and sure use a newly formated uSD - Or you never know when Murphys law can kick in, see my note at the end for a bad luck I had yesterday night
c) Insert the uSD card to your Nook Tablet
d) Copy to your Rooted NOOK TABLET with firmware 1.4.0 the kernel module lbhkvc_km_DEVICE_NOOKTAB.ko which I have provided with in the release folder in the source package uploaded with this message/post.
__may be__ adb push lbhkvc_km_DEVICE_NOOKTAB.ko /data/local/tmp/
or similar or thro the uSD card and a file manager.
e) either from adb (on PC) or a terminal (on NookTab) or serial port (on PC) get to a root shell.
if using ADB on a PC connected to rooted Nook Tab it will be
adb shell
su
NOTE: PC is NOT required, you can also load the kernel module directly from Nooktab by using a android terminal package to get shell access.
f) insmod the copied Kernel module from the root shell/prompt
insmod lbhkvc_km_DEVICE_NOOKTAB.ko
This will load linboothkvc and inturn do a forced reboot/hijack into a x-loader environment, which will inturn load the u-bootk.bin which you had copied into SD card in steps a to c above.
NOTE: Based on what I have verified, u-boot works with out requiring any change to it (unless you want to bypass the security check in u-boot . So ENJOY.
NOTE: This can equally work on a 1.4.1 firmware based Nook Tablet, provided it is rooted. But for that you will most probably have to recompile my kernel module with updated addresses (if it has changed) for the kernel functions which I am using. However if BN hasn't done any change to kernel in 1.4.1 firmware, then the current kernel module which I have bundled it self will work. I haven't verified with 1.4.1 firmware currently (as I haven't upgraded to 1.4.1)
NOTE: Be sure to save any things you might be working on your NookTab, before loading the kernel module, because it will force a reboot with out any mercy, so any unsaved things in your NookTab can get lost. so BEWARE
NOTE: Rather even yesterdays release would have worked 99%, I had a problem with my uSD card being bit corrupt wrt rootdirectory, which is why it was failing yesterday when I finished it originally . However in this release I have also cleaned up the Nirvana code a small bit to avoid the hardcoded ioremap address of UART.
Release v1.1 source with binary for NookTab
Hi All,
Being bit lazy I had not called cache flush once again after disabling the cache, because I was calling it just a wee bit before disabling it and was also expecting any body who enables Cache in Nirvana environment to do a thorough job of cache invalidation before enabling it again.
However found that the linux kernel 2.6.35 used by Nook firmware has this problem, so now I take care of calling cache disable once again after disabling the Cache to be on safe side and it does help for NookTab (while on Beagle and Pandboard I didn't have this issue, it was a newer kernel which I had tried there so may be it has some additional effort put in wrt cache invalidation or it was pure luck, either way haven't debugged that aspect now).
So with this release, linboothkvc provides a better Nirvana environment, except for the SMC related stuff, which I am ignoring for now . So related to that there is some cleanup or bypassing required to be done in Kernel otherwise linux kernel should be pretty much ok in linboothkvc Nirvana environment, haven't had time to look at it fully yet. Will spend sometime in a day or two when a holiday is coming my way.
Check out the README file some of the same info as above and may be a few more things here and there. Also the new binary for NookTab is there in the source package within the release directory.
I've read every one of these posts and have no idea what is going on but I'm glad that it's all working. Great job!
Sent from my Nexus S using xda premium
Getting Linux kernel for NookTab up in LinBootHKVC - Step2 and 3
Hi All,
Step2
------------
There seems to be few race conditions in the linux kernel source and or some initialisations not being done properly in the kernel code, because of which by default the BN kernel source will fail in LinBootHKVC environment.
The below patch fixes 1 such race issue I have noticed wrt initial clock event handler for ipi timer (Have to debug this further later).
NOTE: A initialisation issue with the Cache, in that it not getting invalidated properly during booting was fixed in my v1.1 linboothkvc kernel module, by doing a additional flush before switching to Nirvana. Because that was also a appropriate stuff for linboothkvc also to do, independent of the kernel initialisation issue.
static void ipi_timer(void)
{
struct clock_event_device *evt = &__get_cpu_var(percpu_clockevent);
irq_enter();
- evt->event_handler(evt);
+ if(evt) {
+ if(evt->event_handler) {
+ evt->event_handler(evt);
+ }
+ else
+ printk("WARN:ipi_timer: event_handler missing\n");
+ }
+ else
+ printk("WARN:ipi_timer: evt not set\n");
irq_exit();
}
Beyond this any additional issues, I have to check yet. Also the reason for this race, I haven't debugged for now.
UPDATE1 (Step3)
----------------------
I have further cross checked that CONFIG_OMAP_RESET_CLOCKS causes some problem in linboothkvc Nirvana environment for few specific clocks, either way for now Disabling this config in BN NookTab kernel before compilation will allow the resultant kernel to run SUCCESSFULLY on NOOKTAB.
NOTE: As of now, even thou SMP is enabled, one of the Processors doesn't come up live. That has to be debugged later. But otherwise the Linux is FULLY running in NookTab from with in the LinBoothKVC Nirvana Environment and in turn the Linux User space is also running fine.
That sounds like a big step. Awesome work!
Clean | Stable | Carbon | Flexible | Optimized | Excellent
-> ArchiDroid 2.X <-
Ported to the Galaxy Player 4.0
BIG thanks to @JustArchi
{
"lightbox_close": "Close",
"lightbox_next": "Next",
"lightbox_previous": "Previous",
"lightbox_error": "The requested content cannot be loaded. Please try again later.",
"lightbox_start_slideshow": "Start slideshow",
"lightbox_stop_slideshow": "Stop slideshow",
"lightbox_full_screen": "Full screen",
"lightbox_thumbnails": "Thumbnails",
"lightbox_download": "Download",
"lightbox_share": "Share",
"lightbox_zoom": "Zoom",
"lightbox_new_window": "New window",
"lightbox_toggle_sidebar": "Toggle sidebar"
}
Changelog
ArchiDroid 2.4.3
- Improved archidroid_pixelserv
# Previously archidroid_pixelserv responded to all requests with the same 1x1 NULLGIF response (GIF89a[]![])
# This was OK but in some apps it showed our gif in binary form (GIF89a[]![]) instead of showing nothing, i.e. in Subway Surfers game
# Now we respond with a "NULL" content proper for request
# If app requests JPG, we respond with NULLJPG, if app requests TEXT, we respond with NULLTEXT etc.
# This way app "gets what it wants" and won't show something, which it can't understand to user
# Surprisingly, at least Subway Surfers doesn't show any ad right now, so it also stops some apps from displaying NULL content, instead of showing NULL in binary form
- Removed VNC Viewer
- Updated ArchiDroid Backend tools (dnsmasq, haveged, dnsproxy2, pixelserv) to latest versions
- Updated PA GAPPS to 0417
- Used Carbon as base; Omni is maybe coming in future (if requested; this eventually needs much time)
Download
All Files Dev-Host
Experimentals on GitHub
Stable: ArchiDroid 2.4.3
Oldstable: ArchiDroid 2.X
Remember that you don't need anything else to flash. Google Apps are included already.
Known Issues
All known and unknown Carbon/CM bugs (if any)
Debian and adflash are not working yet (maybe we have to create a virtual ext4 partition cause of too little /data partition).
If this is working it's possible to modify adflash to fit my GitHub repository.
Follow (original) ArchiDroid On XDA!
Write A Review!
Rate the Official Thread!
Buy The Masterchief A Beer (he created this awesome ROM!!)
Like ArchiDroid On Facebook!
Hit Thanks!
Informations:
[ROM] [KVT49L] [OmniROM] [Linaro] [Stable] [Flexible] [Excellent] [20/04/14] ArchiDroid V2.4.3 | Power In Your Hands, a ROM for the Samsung Galaxy Player 4.0
Contributors
JustArchi
andreasltcf
ROM OS Version: 4.4.x KitKat
ROM Kernel: Linux 3.0.x
Based On: Carbon; ArchiDroid
Version Information
Status: Beta
Created 2014-04-28
Last Updated 2014-04-28
[SIZE="+3"]ArchiDroid's FAQ / Q&A (i9300)[/SIZE][SIZE="+1"]Remember.. This is the Galaxy Player 4.0 thread.. And based an Carbon.. It maybe differs from following information..[/SIZE]
[SIZE="+1"]Features / Why ArchiDroid?[/SIZE]
First of all, ArchiDroid includes everything available in it's base. The whole point of ArchiDroid is to improve the base, without needing of making any trade-offs, so by flashing ArchiDroid, you're getting everything offered by the base itself. There's nothing to lose, everything to gain.
You can read detailed information about every ArchiDroid component here. It's a massive wall of text, so I'm only going to list the core features without describing them.
These were written from scratch, they're completely unique and you won't find exactly the same implementation in any other ROM.
ArchiDroid-Unique features:
- ArchiDroid's AROMA Installer
- ArchiDroid's Pocket Debian
- ArchiDroid's Flasher
- ArchiDroid's RunOnce
- ArchiDroid's Init
- ArchiDroid's Backend Control
- ArchiDroid's HArdware Volatile Entropy Gathering and Expansion Daemon (Haveged)
- ArchiDroid's Fast Random Number Generator (Frandom)
- ArchiDroid's Adblock (dnsmasq/dnrd, dnsproxy2, pixelserv)
- ArchiDroid's Forced Update
Apart from that, here, on the credits page, you can find all third-party projects, which have been implemented into ArchiDroid. In addition to that, it's up to YOU to decide if you want to install something, or not.
ArchiDroid focuses on flexibility and user choice.
If you're looking for fastest ROM, choose ArchiDroid.
If you're looking for most battery-saving ROM, choose ArchiDroid
If you're looking for cutting-edge functions, choose ArchiDroid
If you're looking for the most flexible rom ever created, definitely choose ArchiDroid
ArchiDroid adjusts to your needs. You can make it whatever you want. With bunch of presets, modes and questions, you can make your ArchiDroid behave. Check yourself why ArchiDroid is The TOP 1 ROM for Galaxy S3http://forum.xda-developers.com/galaxy-s3#romList, according to number of followers, rates, reviews and downloads count. Check the Reviews, take a look at Video Reviews, do whatever you want to, ArchiDroid is proven to be one of the best ROMs for Galaxy S3, ever created.
Try ArchiDroid once, and you'll never look back. I can assure you.
Disclaimer
Developer's Kitchen
Unless stated otherwise, all ArchiDroid components are licensed under the Apache License:
Code:
Copyright 2014 [email protected]
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
Especially:
ArchiDroid is one of the most complex ROMs ever created. When you start digging in my work, you can very easily get lost. And I'm not talking about base itself, but about everything next to it. You can use every part of my work, as long as:
1. You include proper credit where you should. This usually includes proper comment in a script/file and in the credits of the project, including license (if needed)
2. You let me know about this fact. Through PM on xda or e-mail
I'm always happy to help, especially with the problems I faced in the past. However I also want to be respected, considering that most of the ArchiDroid parts were written from scratch.
Know Your ArchiDroid
ArchiDroid is not only a rom. It's not only a baked android with third-party apps, modifications and tweaks. ArchiDroid is an universal backend which improves rom by many built-in functions.
Let me tell you a story. During developing first version of ArchiDroid 2.0 I experienced many problems, which were not that easy to solve. First of all - apps data. Trust me or not but you can't simply extract data, put it in /data/data after install and hope that it works. Android will detect such injection, report inconsistent of data and wipe everything attached to that. Okay so... How I should provide you with my boeffla preset? CoolTool settings? STweaks profile? If I put my data during flashing it'll get wiped. If I put my data and deny wiping it then Android will report inconsistent of data to user and work unstable. Yes guys, it's impossible to do so without a trick or without recompiling whole rom.
I won't tell you a whole story, because you probably don't want to hear about ArchiDroid development. I'll simply tell you that I overcome MANY difficulties, simply because I do what I like, and I like what I do. After countless number of hours, I can finally provide you with the ROM, which is the best. Why is it the best? Because I'm improving the base, and I'm not making any trade-offs.
GitHub / Versioning / Experimental Versions
You can easily "pack" latest experimental and flash without needing to wait for the next build. I'll tell you how to download and flash it by yourself.
[Newbie Version]
1. Open 2.X-EXPERIMENTAL branch.
2. Click on Download Zip button located in bottom-right corner.
3. Install 7-Zip if you don't have that already. Other programs may, or may not work correctly.
4. Right-Click on zip, select Extract Here
5. Navigate to newly created ArchiDroid-2.X-EXPERIMENTAL folder
6. Select all files WITHOUT __dont_include folder, right-click and select 7-Zip => Add to archive
7. Make sure that you have zip format, Fastest compression (to save some megabytes) and Deflate method of compression.
8. Voila, copy newly created ArchiDroid-2.X-EXPERIMENTAL.zip to your phone and flash as usual.
[Expert Version]
1. Install Git for Windows or Linux
2. If you're on windows then make sure that windows won't change LF into CRLF (git config --global core.autocrlf false)
3. Clone my git repository using .git file. Keep in mind to specify depth, as this repo is really big. (git clone https://github.com/andreasltcf/ArchiDroid.git --branch 2.X-EXPERIMENTAL --depth 1)
4. Select all files WITHOUT __dont_include folder zip them into standard .zip format with Deflate compression or without any compression.
5. Copy .zip to your phone and flash as usual
6. In order to update your local repo execute git pull origin 2.X-EXPERIMENTAL and go to point 4.
Additionally you can watch @JustArchi's short video, which shows how to flash experimental ArchiDroid going through "Expert Version" (Archi's GitHub).
ArchiDroid Features
Know your ArchiDroid, learn how to use it
Introduction / Basics
Welcome to ArchiDroid's world mortal. In this tutorial we will show you what ArchiDroid has "inside" and what it really offers. All of things included in this post are ArchiDroid-specific, which means that you won't find any ROM with the same features, as they're written from scratch.
Here you can find some definitions of the words used in sections below. You should know most of them, but in case somebody is lost here you can get back on track.
Terminal, Shell - Typical android shell, which may be obtained in three ways.
1. Through native Android Debug Bridge (ADB) with command "adb shell" from the PC or even "ADB through network" (if supported).
2. Through any Android terminal app, f.e. Android Terminal Emulator bundled with ArchiDroid.
3. Through secure shell daemon (sshd), which needs to be turned on firstly. This is extremely useful in terms of pocket debian, which will be described later.
You can use any of these methods to access android's terminal, however usually Android Terminal Emulator will be the easiest one, as it's android app bundled with ArchiDroid. WARNING! Most of the commands below WILL require root access. You can log in to super user shell by "su" command right after obtaining shell. If you're unsure if you're logged as root or not, "whoami" command should print actual user, "root" or "unknown uid 0" are OK, others are not.
ArchiDroid's Pocket Debian
From wikipedia:
From debian site:
How exactly this covers our beloved SGS3 (and countless number of other android arm-based phones)?
As you may (or even should!) know, Android operates on it's own Linux-based kernel. Android's kernel is literally a fork of Linux kernel, with a few special/unique functions which are required, mostly hardware-specific. Because of that kernel itself is VERY compatible with everything based on Linux.
However there have always existed one typical problem, lack of proper environment. We have a great kernel, great power, linux-based android environment, but this environment lacks of very common and required libraries/binaries. If you ever wondered what is or what does busybox, this is the answer. Busybox is just a small package which offers a few standalone GNU/Linux binaries, which are required to make certain things work. For example, swap priorities. Android knows what swap is, and nothing else. It doesn't know that swap could have a priority, so if you use android's swapon command on 4 devices, it will firstly fill first device, and then proceed to the next. That's why we need busybox in ALL custom kernels, because android environment isn't enough.
However busybox sometimes isn't enough. If we focus only on Android itself, it is. But if you for example want to run stricly linux-based service, I don't know, a web server for example... Is it possible to run a native linux web server on an android? No, it's not. You should firstly compile such service on arm architecture, including all dynamic and static libraries (wrrr ) in it only to finally get mad because of missing libraries or other dependencies. Of course if you're patient you'd finally compile everything and set up, however how long would it take? A few days maybe? If you're skilled in what you're doing...
This is why I included built-in "pocket" debian in ArchiDroid. It's FULLY compatible with everything compiled/based on armhf/armel GNU/Linux architecture, for example Raspberry Pi. With two easy commands you can literally jump into debian environment and use every typical GNU/Linux utilities known from debian itself. Of course this means nothing for most of the users, because they'll never have any reason to use such debian environment but from the developer side, it's big step forward. The best example is with github. As you know ArchiDroid has it's own repo on github, from where you can download/manage stuff. There also exists git app for linux and windows. If you want to follow "expert" way of flashing experimental ArchiDroid version, such program is required. The scenario is the same as compiling web server for an android, it requires much more effort than it's worth. And even then you can end up with syncing external dependencies and searching for solutions for the problems you've never seen before... And with ArchiDroid's pocket debian? It's as simple as in any debian/ubuntu distro. "apt-get update && apt-get install git" and voila. Your git is installed and ready for work. Going further I've even included git in pocket debian itself. Okay, I have debian, I have git, and what next? With git utility I can for example provide you with delta updates for ArchiDroid! ArchiDroid can easily use pocket debian to set up and sync ArchiDroid's repository and then pack and flash latest version without even needing of a PC, using 7-zip or anything else. Another example? A web server. I know that it's very dev-specific but if you for any reason need a web server running, just for example to test simple website, you can have it with just one command. Going further, VNC? MySQL server? PHP? Python? Perl? Ruby? Maybe conditional tasks with cron? Persistent minimal IRC client? rtorrent with rutorrent GUI over WWW? The list goes on... Anything based on linux will work. You can even host a server for your favourite game, as long as it has armhf/armel binaries (unfortunately most of the games don't).
So that's it. In short, debian is an operating system built-in in ArchiDroid to provide you with (unfortunately missing) GNU/Linux environment, with full power, ready to handle anything you could request. I made my best to include fully working debian in ArchiDroid for a minimal cost. Whole OS is packed in one big tar file, compressed using highest bzip2. As for now pocket debian has ONLY 40 megabytes of size, maybe in future it will have up to 50 megabytes, but no more. It's a VERY small cost for having such great power, especially if you know how to use it.
This is a really cutting-edge feature, mostly because I have no limitiations what I can include in my ROM right now, and while other developers are dealing with OpenDelta updates and many Android-based problems, I'm just launching my pocket debian and manages linux stuff.
I'm SURE that most of the advanced ArchiDroid user will just LOVE this feature, as much as I love it. I'm looking forward to your responses how YOU use pocket debian with your ArchiDroid. It's also a great time to learn what does the debian offer and how you can simplify your common tasks with just one example debian utility .
Technical informations:
1. Pocket Debian does not cause any additional overhead. We don't need to use emulation, neither virtualization to boot our monster. I used chroot technology to "jump" into debian environment with already running kernel and Android. That means additional required CPU/RAM is based on what you run in pocket debian. Booting itself doesn't require anything, just about one megabyte of ram for /bin/bash shell .
2. Android has some restrictions, mostly sockets. It doesn't allow to create inet sockets by default, even for root users. You will need to add your custom debian users to special group called "inet" (GID 3003) to allow creating of inet sockets, and you may also need to add a group to net_raw group (GID 3004) to allow creating of raw sockets. Please keep in mind that it's only required if you're running an app which required it's own socket, for example mysql server. So apt-get install mysql-server will fail right after booting, you will need to use "addgroup mysql inet" and then apt-get -f install to complete installation. Of course "mysql" is the new user under which mysql-server really operates. I've added root to both of these groups by default.
3. The only "real" restriction is the kernel. Our debian uses Android kernel and it's filesystem. It should work with most common tasks but in some cases our kernel may lack specific modules or built-in code, for example tun/tap required for OpenVPN. Still it's enough to run pretty much everything and if you get in touch with your favourite kernel developer you can also kindly ask for specific missing things.
4. Debian is built and included thanks to debootstrap utility, ArchiDroid command used for creating debian environment is debootstrap --verbose --arch armhf --include=git,ca-certificates,ssh,htop,tightvncserver,xterm,xfonts-base --exclude=manpages,man-db,rsyslog,vim-common,vim-tiny testing debian http://ftp.fr.debian.org/debian
HowTo:
Pocket Debian contains two main terminal commands, "adlinux" and "debian". Both of them are described below. By adlinux and debian you boot and jump into debian's chroot, which means you can use any debian-specific commands.
Examples:
passwd - changes password of actual user. This is needed to login as specific user, for example through ssh.
service ssh start - starts local SSH (secure shell) daemon on native port :22, to which you can easily access via any client supporting ssh, f.e. PuTTY. So basicly after you start shell you can literally connect to your local area network (LAN) IP on port 22 f.e. through PuTTY from your PC.
ifconfig - prints network-related informations about online interfaces, including your local IP, which may be useful for connecting to SSH.
htop - Enhanced top utility. Gives you very good terminal-based view on actual running processes, used ram, load, and more.
apt-get update - Syncs with debian's apt repository. This is mandatory to use many of apt commands because ArchiDroid's debian comes without local repo available, however fully configured to download and access it with just one command
apt-get install XXX - installs packet XXX from debian's repository.
apt-cache search XXX - searches for all packets including keyword "XXX". Ultra useful in terms of searching for specific packet.
Please note that pocket debian is VERY similar to normal native Debian/Ubuntu distribution, therefore above commands are not ArchiDroid's magic, they're very widely used in Debian/Ubuntu distros. If you want to learn more, most of the Debian/Ubuntu tutorials will be very helpful.
ArchiDroid's Pocket Debian Booter (adlinux)
You can call "adlinux" command from your favourite terminal.
adlinux is designed to boot and prepare ArchiDroid's Pocket Debian environment. It requires mode to be specified, and also respects any extra arguments passed.
If you call standalone "adlinux" command then it will print usage and then ask you what you want to do with giving proper informations about every choice. Additionally if you know what you want to do, you can also pass arguments directly to adlinux command, f.e. by executing "adlinux default", which will execute adlinux with "default" mode.
Available modes:
default - Will mount /data /system /storage/sdcard0 /storage/sdcard1 and core filesystems in chroot. Default suggested mode
safe - Will mount only core filesystems in chroot. Useful if you don't want to share your storage in chroot
bare - Won't mount even core filesystems such as /proc /dev or /sys. Requires "debian force" to enter chroot. This is the "real" safe mode. You won't be able to interact with an android in any way, while debian itself will work in very limited environment, making core functions unavailable. Suggested only for testing purposes
rebuild - Will automatically reboot your device and remove debian folder in the safe way. WILL CAUSE ALL DEBIAN DATA TO BE LOST!
unmount - Will automatically reboot your device to safely unmount debian environment
Extra options:
extsd - Use external sd card (/storage/sdcard1 /storage/extSdCard), if possible
intsd - Use internal sd card (/data/media/0)
Additional information about modes:
Debian shares core kernel filesystems in "safe" and "default" modes, while it also shares your internal and external sd card in "default" mode. This is nothing to be scared of, as you have full control of what you run in debian, however please note that you CAN'T do whatever you want. All mounted partitions in debian are "binded". "Bind" means that it's mirrored to the mount point and all changes on mounted partition WILL affect the mount point, which is logical. This is nothing to be scared of, as long as you know that debian only extends your environment, it does not fully works in it's own and you CAN cause serious problems from inside of chroot. The only really safe mode is "bare" mode, however in "bare" mode debian can't really do anything, as kernel filesystems are absolutely required for most of the functions. Okay so, you need to know one thing. If you have booted debian you SHOULD NOT touch debian's folder, which is ArchiDroid/debian (on your internal or external sd card, depends what you choosed).. As you know debian for example binds /data to it's folder /data, which is physically ArchiDroid/debian/data. If you for example delete ArchiDroid/debian through root explorer WITH mounted debian then it will ALSO delete debian/data folder, which is binded to /data, and therefore will delete your whole internal sd card, that's why it's extremely important to take care because booted debian becomes part of the android and deleting it can cause at least soft bricks, with a possibility of hard as well. If you want to delete debian folder PLEASE use "rebuild" mode, only through this way you're absolutely sure that nothing bad happens and you won't delete your whole system partition by accident.
Note about extsd option:
Debian requires symlink functionality, typically native windows filesystems DON'T support symlinks, therefore you need to have your external sd card formatted in one of the native linux filesystems, f.e. ext4. adlinux will automatically tell you if debian can be unpacked and used on your external sd card, however it won't be possible under most common filesystems, such as exFAT or FAT32.
Technical informations:
1. Pocket debian archive is located in ArchiDroid/System/debian.tar.gz file. This is "bare" system used for creating environment for the first time, you should not touch it.
2. adlinux detects if debian is already extracted when booting, if not, it's firstly extracted from the file described above.
3. After extracting (if required), core filesystems are mounted with "bind" option based on the mode you've selected in "mode" question above. Typically it mounts /data /system /storage/sdcard0 /storage/sdcard1 /storage/extSdCard /dev /proc /sys.
4. Unmounting is not fully supported right now (linux barrier), therefore both "unmount" and "rebuild" options require a restart to execute properly.
ArchiDroid's Pocket Debian Shell/Chroot (debian)
You can call "debian" command from your favourite terminal.
debian command is designed to allow you "jumping" into debian chroot created by adlinux. Please read how adlinux command works firstly if you haven't done that already. debian command checks if core filesystems are available (if debian is booted), and if they are then it firstly modifies required environment variables to make debian happy (such as TERM, HOME, PATH), then it changes root (chroots) into debian folder, therefore allowing you to execute everything from inside of chroot. It's very generic command, therefore standalone "debian" command won't give you a choice the way adlinux did.
Available options (parameters):
force - required for jumping into bare debian, created with "adlinux bare" command above. This skips debian checks for mounted core filesystems, normally you should avoid it at all cost, unless you know what you're doing. If core filesystems are missing then it's very likely that your debian will be disabled in more than 90%.
extsd - Use external sd card (/storage/sdcard1 /storage/extSdCard), if possible
intsd - Use internal sd card (/data/media/0)
cmd - Executes command in debian chroot
WARNING! cmd parameter will cause all further parameters to be threated as a command passed to debian, therefore you need to make sure that this is the last debian parameter which you want. For example "debian force cmd service ssh start" will skip filesystems checks and execute "service ssh start" in debian's chroot, however "debian cmd force service ssh start" will pass "force service ssh start" to debian, therefore respecting filesystems checks and passing invalid command.
This function is extremely useful for making init.d and other startup scripts. For example you can easily call "adlinux default" and then "debian cmd service ssh start" to call secure shell daemon on every boot with two easy steps.
Technical informations:
1. debian command uses chroot technology to change root of current shell to debian shell.
2. After chrooting to debian directory, /bin/bash shell is automatically called as default debian shell.
ArchiDroid's Flasher (adflash)
You can call "adflash" command from your favourite terminal.
adflash is a great small utility, which allows you to easily update your ArchiDroid to latest stable or experimental version with one easy command and delta upgrade. It utilizes ArchiDroid functions, therefore you must be running ArchiDroid to use it.
If you call standalone "adflash" command then it will print usage and then ask you what you want to do with giving proper informations about every choice. Additionally if you know what you want to do, you can also pass arguments directly to adflash command, f.e. by executing "adflash 2e git", which will execute adflash with 2.X-EXPERIMENTAL version using git mode.
Available versions:
2e - 2.X-EXPERIMENTAL
2s - 2.X-STABLE
1e - 1.X-EXPERIMENTAL
1s - 1.X-STABLE
Extra options:
git - Sets up local git repository, which gives you delta upgrades and bandwidth saving
direct - Downloads targeted branch as .zip file directly from github
clean - Cleans everything up, including local repo and tmp folder from ArchiDroid directory specified below
extsd - Use external sd card (/storage/sdcard1 /storage/extSdCard)
intsd - Use internal sd card (/data/media/0)
nozip - Shows changelog and changes only
Okay so, the most interesting option is the mode...
Direct mode is simple, fast and effective. It downloads target version (stable or experimental) from GitHub server, then it repacks downloaded zip file and makes it available for flash. You should use this mode for one-time downloads, such as once per stable version or two. The only advantage of this method is the ability to download from github (and with one command).
Git mode is complex. It uses ArchiDroid's Pocket Debian (read above) for cloning and updating local ArchiDroid repo. This gives several number of advantages, mostly for using experimental versions. Firstly, by having local ArchiDroid repo you have to download ONLY changes between your snapshot and server's snapshot, which means delta upgrades. Secondly, you have access to all commits from target branch, so you know exactly what has changed since your latest download. Again, this is extremely useful for experimental branch, as changelog may not be up-to-date. Keep in mind that git mode will require additional space on your device for keeping ArchiDroid repository, therefore you sacrifice some space for delta upgrades. This mode is extremely useful for flashing ArchiDroid often, for example daily experimental versions, because in fact you download only new commits instead of whole repo/archive.
ArchiDroid's RunOnce (Backend)
ArchiDroid's Init (Backend)
ArchiDroid's Backend Control
ArchiDroid Backend Control is a set of settings, which controls behaviour of ArchiDroid's Init. It's located in /system/archidroid/dev and contains a number of files, which are recognized by ArchiDroid's Init. You shouldn't directly touch /system/archidroid/dev, instead you can control behaviour of ArchiDroid's Backend through /system/archidroid/scripts. They can be easily executed through any script manager, f.e. Root Browser or Android Terminal Emulator. Some of the settings are also located in /system/archidroid/etc folder, mostly configurations for binaries utilized by ArchiDroid's Init.
ArchiDroid's HArdware Volatile Entropy Gathering and Expansion Daemon (Haveged)
The haveged project is an attempt to provide an easy-to-use, unpredictable random number generator based upon an adaptation of the HAVEGE algorithm. Haveged was created to remedy low-entropy conditions in the Linux random device that can occur under some workloads, especially on headless servers. Current development of haveged is directed towards improving overall reliablity and adaptability while minimizing the barriers to using haveged for other tasks.
The original HAVEGE research dates back to 2003 and much of the original haveged documentation is now quite dated. Recent work on haveged has included an effort to provide more recent information on the project and its applications.
The original research behind HAVEGE use was based upon studies of the behavior of processor caches from a hardware level. The 'Flutter' documents attempt to provide a modern view of HAVEGE at software level through the use of a diagnostic build of haveged that captures the non deterministic inputs to haveged for analysis by external tools.
ArchiDroid has built-in haveged entropy generator. It's controlable through ArchiDroid's Backend Control - ArchiDroid_Haveged_EnableDisable.sh. It's turned on in default configuration, through HAVEGED_ENABLED
ArchiDroid's Fast Random Number Generator (Frandom)
Frandom is a Linux kernel random number generator, which is 10-50 times faster than what you get from Linux' built-in /dev/urandom. And it uses very little (/dev/frandom) or none (/dev/erandom) of the kernel's entropy pool, so it is very useful for applications that require a handy source for lots of random data.
ArchiDroid has built-in frandom activator. It's controlable through ArchiDroid's Backend Control - ArchiDroid_Frandom_EnableDisable.sh. It's turned on in default configuration, through FRANDOM_ENABLED.
Notice: Kernel must support frandom module to actually make use of that. Init will try to search for frandom.ko module and load it, then use /dev/erandom for both /dev/random and /dev/urandom. If your kernel supports frandom, it will work. If it doesn't, obviously this will be skipped even if you have FRANDOM_ENABLED. Check ArchiDroid Init log located in /data/media/0/ArchiDroid/Init.log to check if frandom works properly for you.
ArchiDroid's Adblock (dnsmasq/dnrd, dnsproxy2, pixelserv)
dnsproxy2 is a replacement DNS proxy for Android 4.3+
This currently allows the user to manually override the DNS server IP,
and it sets the correct UID on outbound requests so they can be filtered
via iptables / AFWall+ / DroidWall / etc.
Dnsmasq is a lightweight server designed to provide DNS, DHCP and TFTP services to a small-scale network. It can serve the names of local machines which are not in the global DNS. The DHCP server integrates with the DNS server and allows machines with DHCP-allocated addresses to appear in the DNS with names configured either in each host or in a central configuration file. Dnsmasq supports static and dynamic DHCP leases and BOOTP for network booting of diskless machines.
Dnrd, Domain Name Relay Daemon is a caching, forwarding DNS proxy server. Most useful on vpn or dialup firewalls but it is also a nice DNS cache for minor networks and workstations.
Pixelserv is a super minimal webserver, it's one and only purpose is serving a 1x1 pixel transparent gif file. Using some creative firewalling (netfilter/iptables) rules you can redirect some webrequests (for adds for example) to pixelserv.
ArchiDroid has built-in Adblock. It's controlable through ArchiDroid's Backend Control:
ArchiDroid_Adblock_DnsmasqDnrdModeSwitch.sh
ArchiDroid_Adblock_EnableDisable.sh
ArchiDroid_Adblock_EnableDisableLocalDNSes.sh
ArchiDroid_Adblock_EnableDisableLocalDNSesDaemon.sh
ArchiDroid_Adblock_LockUnlockHosts.sh
ArchiDroid_Adblock_MoabAdawayHostsSwitch.sh
ArchiDroid_Adblock_Reload.sh
It's turned on in default configuration, through:
ADBLOCK_ENABLED
ADBLOCK_LOCAL_DNSES_DAEMON_ENABLED
ADBLOCK_LOCAL_DNSES_ENABLED
ADBLOCK_USE_ADAWAY_HOSTS
ADBLOCK_USE_DNSMASQ
In short. This is a very advanced and powerful solution for blocking ads through DNS queries. First of all we're forwarding all DNS traffic to localhost (127.0.0.1). Then we're handling them through local DNS server - dnsmasq (default), or dnrd (option). Our local DNS server reads blocked hostnames through special /system/archidroid/etc/hosts file, then if no record is found, it forwards DNS query to OpenDNS/Google DNS servers, or if it's found, returns 127.0.0.1 as the address. Lastly, pixelserv is providing a 1x1 NULLGIF response on local web server, so instead of big black/white screen instead of the AD, we get 1x1 transparent pixel, which usually perfectly hides ad from the app or the website.
Extra features:
1. You can specify if you want to use dnsmasq (default), or dnrd (option) as a local dns server. Dnsmasq is more flexible, modern, faster and has less memory footprint, however I also left dnrd as an option, because it's proven to work stable.
2. You can specify hosts file, which you want to use. In default configuration we use AdAway's hosts file, with more than 30 thousand of records, which results in extra ~2.5 MB memory usage. You have also an option to use MOAB (Mother Of Ad Blocking) hosts file, with more than 330 thousand of records, which will result in about ~30 MB memory usage. Eventually you can append your own rules or use non-standard hosts file, available in /system/archidroid/etc/hosts. Pro tip: You can point AdAway to use this hosts file (/system/archidroid/etc/hosts_adaway), which will result in automatic updates. /system/archidroid/etc/hosts is a symbolic link, either to hosts_away or hosts_moab, if you want to specify your own hosts, you can delete symbolic link and write your own rules.
3. Original /system/etc/hosts file has been locked from editing. This is to ensure that AdAway or other adblockers won't use obsolete and slow method of blocking ads through hosts. The whole point of implementing Adblock in ArchiDroid is to provide you with super-fast, flexible and effective way of blocking ads, also with getting rid of black/white ad screen. In 99% situations you don't want to touch ArchiDroid's default behaviour, as it blocks ads perfectly. Eventually, if you have a very good reason, you can unlock original hosts file through ArchiDroid's Backend Control and modify them, however keep in mind that every additional rule WILL slow down your network speed.
4. In default configuration local dns server uses two OpenDNS servers at port 5353, two Google DNS servers at port 53 and up to two local DNS servers provided by your Wi-Fi/3G connection, which overall gives a sum of 6 remote dns servers. In some rare scenarios (f.e. some wi-fi hotspots) you can notice that a moron, administrator of this wi-fi, blocked all dns queries and forces you to use his DNSes. This is BAD because of freedom and so on, but it's very common practice, that's why I turned on local DNSes as well. If you want to improve your privacy at least a bit, you can disable local DNS servers and then use only OpenDNS and Google DNS.
5. Above option initialy has been written to allow you one-time access to such non-trusty wi-fi's. But if you for any reason need automatic update of your local DNSes (3G and Wi-Fi's will use different local DNSes), you can also turn on Local DNSes Daemon, which will automatically query and update local DNSes if needed. This is also turned on in addition to local dnses above, of course in default preset.
ArchiDroid's Forced Update (RunOnce)
Forced update selected during mode selection in aroma tells RunOnce to work in "INSTALL" mode even on "UPDATE" mode, apart from that it works exactly the same as update mode, only RunOnce is affected.
Credits
ArchiDroid Core
- AROMA Installer
- AROMA Filemanager
- PhilZ Touch Recovery
- SuperSU
- Nova Launcher
- TouchPal Keyboard
- Hacker's Keyboard
- Android Terminal Emulator
- BetterBatteryStats
- Cool Tool
- Greenify
- MX Player & Custom Codec
- LMT
- Root Browser
- Titanium Backup
- CrossBreeder
- Online Nandroid
- Xposed Framework
- App Settings
- XPrivacy
- Debian
- cURL
- GitHub
ArchiDroid 2.X
- Carbon Rom
- Linaro Toolchain
- Impulse Kernel
- Spirit 2
Special thanks to:
- @JustArchi for creating ArchiDroid icluding all it's optimizations, a helpful hand to make this possible and his AWESOME SCRIPTS + documentation inside them.. really.. this guy is able to do ALL with scripts..
- @zaclimon for his work on our device with its own very little but awesome community, Impulse kernel, his sources for building and again all his contributions for our device..!!
- Kenshin, for graphic design and ArchiDroid Touhou bootanimation
- @mrtur, for graphic design and helpful hand during ArchiDroid experimental tests
- @malachow, for helping users across both international and polish board, sharing the spirit of ArchiDroid
- All ArchiDroid Contributors, for improving and making ArchiDroid better!
- ArchiDroid Facebook Group, for beta-testing the very first alphas of ArchiDroid 2.0.0
- ROM Cleaner, for awesome generic list of bloatware
- Android Revolution HD, for being ex-ArchiDroid 1.X base
- WanamLite, for being ex-ArchiDroid 1.X base
- Temasek's Unofficial Build, for being ex-ArchiDroid 2.X base
- crDroid, for being ex-ArchiDroid 2.X base
- You, for choosing ArchiDroid over other available ROMs
Nice to see it here as well, good luck with the port .
okay.. if anybody out there using this rom.. pocket debian is working fine.. (img on external sd).. have to write the scripts to do it all automatically..
edit: ups.. have to upload the rom first.... but if there's any interest in this.... btw if you're using carbon just now.. you'll love this rom..
I think people would be interested... I would be interested in trying it out once I get more time on my hands, maybe use it as a daily driver if it fits my needs.
I'm currently in the process of creating a custom version of AOSP that will edit the functionality of the Host-Card Emulation feature of Android, allowing me to manually set the AID produced when my device goes into emulation mode.
I've downloaded the Android source, and have located the files I think governs HCE functionality.
From the NFC System Service, I've located the following files:
"CardEmulationManager.java" and "HostEmulationManager.java" located in "/aosp/packages/apps/Nfc/src/com/android/nfc/cardemulation/"
And from the Android core framework, for the NFC API I have found:
"CardEmulation.java" located at "/frameworks/base/core/java/android/nfc/cardemulation/CardEmulation.java"
From here, i'm unsure how to get started in editing the code. I'm unsure of the interactions between the files, and how they are called at run (I.e. which functions in the file I should be looking at, how would I go about adding my own code/modifying existing code in the file)
Is there any documentation online about how these files work/are ran? Or how would I go about finding out on my own (Through debugger tools etc.)
I'm currently using Eclipse for Ubuntu, and will be developing for a Nexus 7
Hi!
This module may be mainly interesting for developers. This module alone does nothing if not properly used together or within another app. Only use it if you know what you are doing.
Overview:
I created a Xposed module which overrides the Access Rule Checks within the SIMalliance Open Mobile API. These checks normally determine which app is allowed to access a SIM-card based Secure Element (SE) and which is not. Normally within the Secure Element (SIM card) there exists a special "Access Rule File" (also called "ARF") or a special card applet called "Access Rule Application Master" (also called "ARA-M") which has the AID A00000015141434C00) and (basically) contains application signatures and according access rules. So the maintainer of the Secure Element can define (and also update) the access rules with these mechanisms: which app is allowed to access the SE and which not. (see reference [1] for details below).
So while the access rules are securely stored in the hardware module (SE) the enforcement of these rules is done in software (within the Open Mobile "SmartcardService.apk"). And this is where this Xposed module hooks into. By using this module (or by integrating it into your app) you may access the SIM-card-based Secure Element with your app, even if the access rules within the SE would not allow it.
Here's the code:
https://github.com/johnzweng/XposedOMAPIOverrideSEAccessRules
This module does not globally disable the access rule checks but instead only grants full access to a single package name (app) which you manually have to define in code as TARGET_APPLICATION_PACKAGE_NAME.
More details and background story:
A few months ago the banks here in Austria started to roll out NFC-based tap'n'pay solutions which allow you to pay with your Android smartphone worldwide at every NFC-capable payment terminal. In contrast to Android Pay this is not a cloud-based software solution (using Host Card Emulation) but instead really uses a hardware-based secure element (like the chips used within EMV (chip'n'pin) plastic NFC banking cards). Austrian Banks decided to use Secure Elements located within the SIM-card (which is basically the same technology as your banking-card chip) so they don't need cooperations with all the phone manufacturers (and also not with Google), but only with the three mobile network operators (MNOs) here in Austria (which control all the SIM cards).
Unfortunately Google has not included an API in current Android which allows accessing SIM-card-based Secure Elements from an app (I guess they don't want banks and MNOs to develop their own independent payment solutions but instead want to push Android Pay). For this reason one of the largest smartcard manufacturers (Giesecke & Devrient - G&D) started to work on a third-party system API which is nowadays known as SIMalliance OpenMobile API (OMAPI). The SIMalliance is a group of industry players which want to push the use of Secure Elements in mobile phones. See also: Members of SIMalliance.
As this API is not part of official Android API the phone manufacturers (OEMs) have to include this API additionally on the phones. A lot of manufacturers do this already. You can check on your phone if this 3rd-party API exists by looking for these files:
/system/etc/permissions/org.simalliance.openmobileapi.xml
/system/framework/org.simalliance.openmobileapi.jar
/system/priv-app/SmartcardService/ (which hosts "SmartcardService.apk")
(and optionally in newer versions: /system/priv-app/UiccTerminal/)
As Google currently doesn't support this type of access to the SIM-card Secure Element you will not find this API on the Nexus phones. One exception was the Nexus 6 running Lollipop where Google included the SIMalliance Open Mobile API because they supported the Softcard (formerly Isis Mobile) wallet. After Google has acquired Softcard in 2015 they removed again the OpenMobile API from Nexus 6 in Android 6.
As I personally used a LG Nexus 5 and now use the Motorola Nexus 6 I started to work on integrating the Open Mobile API myself (which should be possible now on every phone since Android Lollipop (API level 21) as the TelephonyManager system class has got a few interesting new methods including one for sending APDUs over a logical channel to the SIM card: iccOpenLogicalChannel(String AID). As every phone running Android 5 or newer must implement this API you now can get the OMAPI working on every phone running Android 5 or newer (with minor restrictions). (Also the phone needs to have a special hardware wiring between the SIM card and the NFC chip - see "Single Wire Protocol" for details - to get a NFC payment working). But this is a different story which doesn't belong in this thread. For the interested, look at my Github repo which contains a fork of OMAPI working on unmodified Android versions (and two pre-built releases for OMAPI 2.05 and OMAPI 3.0 under the "releases" section - including a short How-To). This worked for me on a Nexus 5 and a Nexus 6.
But back to this topic:
After I had worked out the integration of OMAPI into Android 6.0.1 I finally was able to use tap'n'pay with my Nexus 6 using the banking card within my SIM card Secure Element. But as I am curious I was also interested in exploring my banking-card within the SIM Secure Element using a self-written app. This was when I realized that there is some kind of access control within the Secure Element which blocks my own test-app but not the app of my bank. (See reference [1] for details on these access rules.) To circumvent this I wrote this Xposed module and voilá, now we also have the possibility to talk with the Secure Element over OMAPI.
Final notes:
This Xposed module will only work if your device has the SIMalliance Open Mobile API (OMAPI) installed
This Xposed module may not work if the OMAPI on your device has been compiled using code obfuscation or was modified otherwise by your phone OEM (as the OMAPI is not part of official Android, every OEM may include its own version)
This Xposed module also may not work on other versions of OMAPI (it was tested with OMAPI v2.05 and should also work with OMAPI V3.0)
I hope this might be useful for someone. Have a nice day.
References:
[1] GlobalPlatform Device Technology - Secure Element Access Control (PDF)
[2] Open Mobile API specification - V2.05 (PDF)
[3] Open Mobile API specification - V3.0 (PDF)
[4] my Github repository for this Xposed module: XposedOMAPIOverrideSEAccessRules
[5] my Github repository with OMAPI fork for working on unmodified Android 5 or newer
I'm very impressed! Love your 'investigations'
I installed OMAPI 2.05 with adb on my xperia sp with cyanogenmod 13 (android 6.0.1).
After that I installed the elba-pay app, but it's not working. May you help me?
polo_joe said:
I installed OMAPI 2.05 with adb on my xperia sp with cyanogenmod 13 (android 6.0.1).
After that I installed the elba-pay app, but it's not working. May you help me?
Click to expand...
Click to collapse
Hi polo_joe:
This doesn't directly match the topic of this thread. But send me a PM then we can continue to communicate on a different channel (email, etc.). Most interesting would be a output of "logcat":
On your computer enter the command: "adb logcat -v time > logcat_debug.txt" in a terminal window then try to start the ELBA Pay app and afterwards look in the "logcat_debug.txt" logfile for errors.
john
androcheck said:
Hi polo_joe:
This doesn't directly match the topic of this thread. But send me a PM then we can continue to communicate on a different channel (email, etc.). Most interesting would be a output of "logcat":
On your computer enter the command: "adb logcat -v time > logcat_debug.txt" in a terminal window then try to start the ELBA Pay app and afterwards look in the "logcat_debug.txt" logfile for errors.
john
Click to expand...
Click to collapse
thanks, will do!
Hey Guys,
I'm working on a global audio tuning system for Android similar to Viper4Android and JamesDSP, but allowing the User to program and create their own audio filters on the fly. In order to do so I need to link against libLLVM.so which is one of the Android Platform Libraries which doesn't seem to be exposed via the NDK (https://android.googlesource.com/platform/external/llvm/).
My assumption is that I will need to build that library from AOSP Source independent of my project and then link against my locally built copy. Once compilation of my project is complete I don't need to distribute my locally built libLLVM.so file since I can leverage the one existing on the actual device.
I just wanted to see if anyone has any experience doing this, or if I'm going off the deep end.
Thanks!