Related
I have been following the Android scene here pretty regularly but the os I have always really wanted is a full fledged Linux. I heard about, and wanted a N900 because of Maemo but I'm too poor to buy one . I then heard about the Mer project https://wiki.maemo.org/Mer to port Maemo to other devices and saw it booted on the kaiser http://forum.xda-developers.com/showthread.php?t=565480. On page two of that thread it is stated that someone (Mdrobnak) got it up and running pretty well on his Fuze with a couple tweaks to Xorg.conf, the kernel, and to the splash screen (resized to 640x480). I've been working over the past few days on getting it to start on my fuze but haret keeps giving errors about pc_clk_disable. I resized the splash, changed Xorg.conf to be 640x480 and did the rest of the directions on the Kaiser thread, but no luck. I would love to get a port of this going, as being able to use ~95% of Ubuntu packages working on my phone would be awesome. I wasn't quite sure how to patch the kernel; that would be the zImage, right? Any suggestions?
I'm going to be on #mer and #htc-linux irc on freenode quite a bit until we get this working well if anyone wants to discuss things, not that I can contribute much.
Would it be possible to create a rootfs.img file similar to how XDANDROID does it for people too lazy to partition their card or just plain suck like me?
Sounds interesting! Keep us updated!
Oh god! Ill be in 7th heaven if it's going to be done!
I think you can use the zImage of xdandroid, maybe it'll work
I tried the xdandroid zImage but it didn't work. This is really a call for help. I really don't know what I am doing, lol. I'm not even close to being a dev. Try out the steps on the Kaiser thread but use the stable 0.16 distribution as the testing links don't work. See if you can figure anything out.
Maemo on the Raph ? that would be great! maybe we could run it in dual boot with android and abandon WM for ever !
To get X working, use glemsom's kernel, and add msm_fb.fix_x=1 to cmdline.
For touchscreen, you might want to add msm_ts.raw=1 (but it can also work without it)
phhusson said:
To get X working, use glemsom's kernel, and add msm_fb.fix_x=1 to cmdline.
For touchscreen, you might want to add msm_ts.raw=1 (but it can also work without it)
Click to expand...
Click to collapse
Thanks, phh. Will try it when I get home.
that's the old and ugly msm_fb refresh hack.
I will update here once I have xorg working on kovsky and kaiser.
Man, I actually feel like this is getting places. Got Phhusson from XDANDROID and dcordes from the original Android on Diamond thread. Quite the star power.
@dcordes: What needs to be edited to use VGA; just the first screen resolution dimensions in Xorg.conf?
what image are you using?
I'm using the Smartq5 from here: http://wiki.maemo.org/Mer/Releases/0.16. My main problem is with the kernel though I think; haret hangs after about 15 seconds; I'm gonna create a log of it.
On my Raph300 I can login on the terminal using the Q5 image. I was using the latest zimage from gelmsom. But kernel is screaming some stuff a bout a clock not having an ena bit and something about an rpcrouter being blocked. When I try to start X, I get an error loading libpicacces.so.0
I get that same error, but Haret hangs when I get it. Please share what you did: e.g. editing xorg, exact zImage used, what you used to partition, your default.txt, whether you edited the splash screen image etc. I'd like to get a database of sorts of people's configurations. Good to know though. Did you add Phhusson's tip to default.txt?
"To get X working, use glemsom's kernel, and add msm_fb.fix_x=1 to cmdline.
For touchscreen, you might want to add msm_ts.raw=1 (but it can also work without it)"
Yes i added the msm_fb.fix_x=1 to the cmdline. Unfortunatly it is not the solution for this problem. We have got some trouble with the libraries. Maybe something is missing.
LordKiwi said:
Yes i added the msm_fb.fix_x=1 to the cmdline. Unfortunatly it is not the solution for this problem. We have got some trouble with the libraries. Maybe something is missing.
Click to expand...
Click to collapse
Last time I tried it worked automatically (I mean for X video output.) with msm_fb.fix_x=1
Sorry it works. JesusFreak and I extracted the tar file the wrong way.
I should have type tar -pxvf. I forgot the p which is very important.
Now it stops with a different error.
Code:
(**) FBDEV(0): using shadow framebuffer
(II) Loading sub module "shadow"
(II) LoadModule: "shadow"
(II) Loading /usr/lib/xorg/modules//libshadow.so
(II) Module shadow: vendor="X.Org Foundation"
compiled for 1.6.0, module version = 1.1.0
ABI class: X.Org ANSI C Emulation, version 0.4
(EE) FBDEV(0): FBIOPUT_VSCREENINFO: Invalid argument
(EE) FBDEV(0): mode initialization failed
Fatal server error:
AddScreen/ScreenInit failed for driver 0
Dang it; I just can't get it to work. I'm the thread starter and I can't do it, lol. Here's what I did:
1. Downloaded 0.16 stable smartq5 rootfs.
2. Extracted the tar to my home folder. (Now that I think of it; I did it with archive manager not the command line, so no -p setting or anything. That could be the problem.)
3. Edited the xorg.conf file to change the resolution to 640x480.
4. Changed the splash screen size to 640x480.
5. Formatted my sd card to ext2 for 1.5 gb and fat32 for .5 gb using command line following these directions for ext2: http://wiki.maemo.org/Mer/Documentation/SmartQ_Installation_for_the_Windows_user and gparted for fat32.
6. I then used the directions on the same pagee as before to copy the file, except using cp instead of mv.
7. I copied haret, a zImage, and the startup.txt that I renamed to default.txt and edited to the fat32 partition.
8. I started Haret.
I attached my default.txt.
The formatting is done correct. You should try to use the tar command line wich you can find on the same page. It should solve the problem of loading the library.
•Type sudo tar -pxzvf *.gz and press return. This will now untar the entire 'rootfs' to the card. (Install Mer).
LordKiwi said:
Sorry it works. JesusFreak and I extracted the tar file the wrong way.
I should have type tar -pxvf. I forgot the p which is very important.
Now it stops with a different error.
Code:
(**) FBDEV(0): using shadow framebuffer
(II) Loading sub module "shadow"
(II) LoadModule: "shadow"
(II) Loading /usr/lib/xorg/modules//libshadow.so
(II) Module shadow: vendor="X.Org Foundation"
compiled for 1.6.0, module version = 1.1.0
ABI class: X.Org ANSI C Emulation, version 0.4
(EE) FBDEV(0): FBIOPUT_VSCREENINFO: Invalid argument
(EE) FBDEV(0): mode initialization failed
Fatal server error:
AddScreen/ScreenInit failed for driver 0
Click to expand...
Click to collapse
I'm an idiot....... I saved my startup.txt as startup.doc. That's why X didn't start.
0.17 release
Is possible run the 0.17 realease in raphael more fast than in N810?
kaiser ubuntu port based on zubuntu by Omegamoon for Sharp Zaurus was successfully launched on HTC Nike!
Instructions:
1. download kernel
http://84.23.71.227/kerneloftheday/zImageUbuntu
2. download ubuntu basefiles and rootfs
3. place ubuntu.img, haret, initrd.gz and zImageUbuntu to Storage Card/ubuntu/
4. place this default.txt there too
Code:
set RAMADDR 0x10000000
set MTYPE 1724
set KERNEL zImageUbuntu
set initrd initrd.gz
set cmdline "pm.sleep_mode=1 mddi.width=320 mddi.height=480 lcd.density=160 board-htcnike-keypad.keypadlayout=1 board-htcnike-keypad.sticky_timer_time_ms=350 no_console_suspend"
boot
5. change board-htcnike-keypad.keypadlayout=1 to 0 if you have 20key nike.
6. run haret and press "Run", ubuntu & kernel will be loaded to login prompt, login with username root and no password, you will get root shell.
7. write
Code:
export TSLIB_TSDEVICE=/dev/input/event0
ts_calibrate
and calibrate your screen.
8. write startx and see LXDE running on your nike! you've got real linux box in pocket!
Note, rootfs for ubuntu needs 1GB of free space on your sdcard!
For now, you cannot work with screen, we have some difference from kaiser panel's, i.e. ubuntu gui is completely unusable, this thread were started to find out why.
based on this thread from kaiser forum.
Credits: dzo, Omegamoon, domy007, mblaster and guys from Rhodubuntu thread.
I dont get any response when i try to calibrate screen on niki 200
thats strange, it must work.
I will debug on touchscreen problems, but ubuntu running and that is great, now we have only input-devices problems, and this is more better than nothing.
Sounds great, but wouldn't it be better to try something smaller like puppylinux or Feather linux ?
zubuntu is only we have compiled for ARM and working with our hardware that starts X and tested on other htc's devices.
rzk333 said:
zubuntu is only we have compiled for ARM and working with our hardware that starts X and tested on other htc's devices.
Click to expand...
Click to collapse
Ok thanks for the information, did some reading before and was always wondering why they use ubuntu but this answer set things straight.
Greetz,
Great to see a full linux distro around for htc devices. I will fix the '_' tomorrow... Any other missing characters?
How is the speed of ubuntu? Is it very sluggish or usable?
'=' character maybe, for export line.
zImageUbuntu is updated (your link is still valid).
'=' and '-'are on '*'-button, '_' is Alt-
I have included the msm_fb_refresh and the keypad changes into my auto-updated-kernel, so you should be able to use the fresh kernels from my thread with ubuntu if you want recent updates (zImageUbuntu is not automatically updated).
got some progress,
1) msm refresh thread seems to disable sleep mode in android, thread polls framebuffer many times in second and drains battery.
2) linux logo on bootup must be disabled - logo gets over ts_calibrate's first crosshair point.
2) still trying to make touchscreen and tslib friends - i2c bus debug messages logs saying that touchscreen is connected and recieves data correctly.
found some data:
/dev/input/event0 - htcnike-kbd
/dev/input/event1 - htcnike-ts
but still no data from touchscreen...
rzk333 said:
got some progress,
1) msm refresh thread seems to disable sleep mode in android, thread polls framebuffer many times in second and drains battery.
2) linux logo on bootup must be disabled - logo gets over ts_calibrate's first crosshair point.
2) still trying to make touchscreen and tslib friends - i2c bus debug messages logs saying that touchscreen is connected and recieves data correctly.
found some data:
/dev/input/event0 - htcnike-kbd
/dev/input/event1 - htcnike-ts
but still no data from touchscreen...
Click to expand...
Click to collapse
Disabled msm refresh thread from the android kernels again. The kernel you linked in your first post is now automatically updated with anything new in the repository. Bootlogo is disabled, too. Good luck with tslib, let me know if when you need something else to be hacked into the kernel.
mblaster
I've just test the usb ether function in the zImage from the post
showthread.php?p=6675397
and it works just fine.
(Just when I had managed to make adbd works... but ssh is more reliable)
Thanks gTan64.
wow, that is good, now I can dump system state without hours of keyboard typing
I've managed to get internet by usb (cdc only) thanks to gTan64: just apply the patch from is post ("Debian on the vogue"), you can extract it from the download links.
And the TOUCHSCREEN WORK!!!!!!!
But you need to compile tslib from the tslib git (github.com/kergoth/tslib.git), the one in the rootfs from this post doesn't work.
Will make a more complete post later, I think.
Bye.
Ubuntu always shows me a black screen when I try to startup it!
What can I do make it work?
omg, I knew that something is wrong with tslib in this rootfs!
thanks for r&d, if you will make some packages & mans, I'll add them to first post.
Maybe you can upload some pics ?
But you need to compile tslib from the tslib git
Click to expand...
Click to collapse
I tried to compile - but no luck, autoconf fails to make configure...
I also uploaded debian rootfs and posted links in gTan's Debian on Vogue thread.
nothing happened last month?
I am working on an open source RTL support for Android and I need help testing...
I have attached a normal app (that should work on any Android 1.5 and up) that runs my method before calling drawText.
The purpose is to make drawText call to it so it will support BiDi.
It reads a file called /sdcard/test.txt from which it gets the test string.
It does not support newlines.
Try it and tell me what you think.
The reason I don't publish the source yet is because knowing how it works would prevent it from being tested properly.
I hope I'm putting this in the right forum...
Hebrew Supported...
Is it also, support the font for Hebrew?
Because in ver 2.1, the have Hebrew but sometimes you see square or ? signs.
TheAgent1982 said:
Is it also, support the font for Hebrew?
Because in ver 2.1, the have Hebrew but sometimes you see square or ? signs.
Click to expand...
Click to collapse
The fonts exist, there's no work necessary on them, you have to push them to the device and that's it...
biditest with Android 2.2 FroYo (FRF50) Arabic test.txt
I tested with Android 2.2 FroYo (FRF50)
without /sdcard/test.txt and with my own text
I don't know about the Hebrew, the Arabic text are not joined (re-shaped) correctly
dudyk said:
The fonts exist, there's no work necessary on them, you have to push them to the device and that's it...
Click to expand...
Click to collapse
I have TP2 running Android 2.1 .
So, any idea how to do it?
Thanks for writing an open source BiDi support.
I did a very short test:
1. It did not render the whole line of text.
2. It looks like it has problems when switching from one language to another. E.g. the closing bracket after the hebrew word is rendered as open.
I attached the original text files and screen shots.
In addition I attache a screen shot of TxtPad Light and how it is rendered there.
TheAgent1982 said:
Is it also, support the font for Hebrew?
Because in ver 2.1, the have Hebrew but sometimes you see square or ? signs.
Click to expand...
Click to collapse
This might be a problem of character encoding.
I did a test with a UTF-8 file and a Hebrew ISO 8859-8. The first rendered the letters correct but the second showed only question marks. (See picture)
As alternative keyboard I recommend using either AnySoftKeyboard or SmartKeyboard.
It is in CM-5.0.8, with a fix on the issue list.
Dear Dudy,
Will your fix work for 2.1 as well? Can you give directions how to apply it to other ROMs? I own an HTC Desire, and would love to implement the fix. So far I just have the common StaticLayout fix.
Thanks,
Ron
EDIT: More specifically, it would be great if you can tell us which files to take from the CM update with the help of baksmali and smali in order to graft them into other distros. That's the way I implanted StaticLayout.smali in MoDaCo roms.
CM 5.0.8 is 2.1. It should apply to any 2.1/2.2 rom, just take my 4 commits from CM's git from framework base repository.
dudyk said:
CM 5.0.8 is 2.1. It should apply to any 2.1/2.2 rom, just take my 4 commits from CM's git from framework base repository.
Click to expand...
Click to collapse
Thanks for the reply, Dudy. I'll be more specific about what I am trying to do: I am trying to pull the files with your patches from the CM-5.0.8 ROM without compiling anything.
With the StaticLayout fix, I used baksmali and smali to disassemble the classes.dex from framework.jar and replace StaticLayout.smali. This worked.
When I look at the page github.com/dudyk/android_frameworks_base/commit/1b0aca31c3e03a5a323276cd15a8df4203a1792c - the one with your commits (is that the right one?) I cannot figure out all the files that I would need to replace. Sorry, I am not a programmer...
I can pretty much figure out that all the files preceeded by core/java/android/ will be found with their exact names in the classes.dex from framework.jar. But I have no idea where to locate the compiled files that resulted from AndroidGraphics2D, the three Canvas files and the ResourceTypes.
If you can point me at the right direction I'll be able to convert a script that was used for the StaticLayout fix so that anyone with a Mac or Linux would be able to easily implement your fix to any none-odexed distro.
Thanks so much in advance!
you didn't add U+0600 support in your Canvas.java @ cyanogen's git
PapaDocta said:
you didn't add U+0600 support in your Canvas.java @ cyanogen's git
Click to expand...
Click to collapse
Why do you say that? I tested for U0590 to U07B1, isn't that enough?
my bad.. i don't know how i overlooked the U07B1.... it's more than enough
Dear Dudy, I really need your help.
Back in the old days (a month ago), before your wonderful work, the best fix we had was Omri Baumer's StaticLayout fix. I was able to use the baksmali and the smali utilities to extract the smali file from framework.jar of CM, and graft it into ANY phone. Another user on the iAndroid forums created a script that automated everything.
I wanted to do the same with your fix. So I compiled a vanilla 2.1 and then applied only your patches, manually, and recompiled. I used baksmali to disassemble all the framework files, and discovered that the changed files were:
android/graphics/Canvas.smali
android/graphics/Canvas$EdgeType.smali
android/graphics/Canvas$VertexMode.smali
android/text/Layout.smali
android/text/SpannableStringBuilder.smali
android/text/Styled.smali
android/widget/TextView$CharWrapper.smali
The two additional Canvas files were tagged as changed, but no textual difference is evident. So altogether we are talking about 5 files that ideally would be easily grafted into ANY device running 2.1, without recompilation.
The problem was that the new Canvas.smali file caused boot loops on my HTC Desire. Logcat caught this:
Code:
D/AndroidRuntime( 103): --- registering native functions ---
W/dalvikvm( 103): Unable to register: not native: Landroid/graphics/Canvas;.drawText (Ljava/lang/String;FFLandroid/graphics/Paint;)V
E/JNIHelp ( 103): RegisterNatives failed for 'android/graphics/Canvas'
E/AndroidRuntime( 103): Unable to register all android natives
This did NOT happen when I tried the vanilla Canvas.smali. So it must be something in your patches. If you can help me figure out how to fix this, your fix will be easily implemented into any un-odexed 2.1 device. The script is all reworked and ready... Just this problem needs ironing out.
Thanks so much - for your wonderful work, and for taking the time to read this.
Ron
ClassicalDude said:
I wanted to do the same with your fix. So I compiled a vanilla 2.1 and then applied only your patches, manually, and recompiled. I used baksmali to disassemble all the framework files, and discovered that the changed files were:
android/graphics/Canvas.smali
android/graphics/Canvas$EdgeType.smali
android/graphics/Canvas$VertexMode.smali
android/text/Layout.smali
android/text/SpannableStringBuilder.smali
android/text/Styled.smali
android/widget/TextView$CharWrapper.smali
The two additional Canvas files were tagged as changed, but no textual difference is evident. So altogether we are talking about 5 files that ideally would be easily grafted into ANY device running 2.1, without recompilation.
The problem was that the new Canvas.smali file caused boot loops on my HTC Desire. Logcat caught this:
Code:
D/AndroidRuntime( 103): --- registering native functions ---
W/dalvikvm( 103): Unable to register: not native: Landroid/graphics/Canvas;.drawText (Ljava/lang/String;FFLandroid/graphics/Paint;)V
E/JNIHelp ( 103): RegisterNatives failed for 'android/graphics/Canvas'
E/AndroidRuntime( 103): Unable to register all android natives
This did NOT happen when I tried the vanilla Canvas.smali. So it must be something in your patches. If you can help me figure out how to fix this, your fix will be easily implemented into any un-odexed 2.1 device. The script is all reworked and ready... Just this problem needs ironing out.
Thanks so much - for your wonderful work, and for taking the time to read this.
Ron
Click to expand...
Click to collapse
My changes add a native method (a C++ one) to the sources, from what I know, it is in the dex file, but maybe smali's files do not extract it, or extract it differently.
I never tried to disassemble files in android and reassemble them, so I have no idea how to help you besides describing the changes to you.
P.S.
Do you think that HebVillian is using my fix this way? I believe that I'm the first to fix this for ROMs above 2.0 (BTW, it's in the froyo branch of CM as well).
I don't know about HebVillain. The maker of the rom was made aware of the way I patched StaticLayout, so it may well be. Perhaps he also read your thread and applied the actual patches...
I noticed that in one of your commits:
http://github.com/dudyk/android_fra...0aca31c3e03a5a323276cd15a8df4203a1792c#diff-5
You added "native" to drawText. It is exactly the function in that line that is throwing the error on boot - about not being native. I tried recompiling with that particular change reversed, but it did not make a difference. Can you investigate the matter? Perhaps the way the Nexus 1 boots is different than that of other devices, and the checks are not as strict. If this is indeed a code problem, perhaps it needs to be addressed for the patch to be viable for other devices.
Take everything I say with a grain of salt - I am no programmer. Just following what I read and understand.
I have no idea if this helps, but googling the words
android registernatives
results in quite a few technical posts complaining about similar problems...
I am sorry for being a nag. But I hope I am getting closer to the source of the problem.
The Android emulator was able to boot up just fine with the patches applied. The logcat stated at the beginning of the runtime:
Code:
D/AndroidRuntime( 29): >>>>>>>>>>>>>> AndroidRuntime START <<<<<<<<<<<<<<
D/AndroidRuntime( 29): CheckJNI is ON
I/ ( 30): ServiceManager: 0xad08
D/AudioHardwareInterface( 30): setMode(NORMAL)
I/CameraService( 30): CameraService started: pid=30
The very same smali files, when applied to the HTC Desire's framework, generated the native error I quoted before. But logcat shows that the beginning of its runtime is indeed different:
Code:
D/AndroidRuntime( 197): >>>>>>>>>>>>>> AndroidRuntime START <<<<<<<<<<<<<<
D/AndroidRuntime( 197): CheckJNI is OFF
D/dalvikvm( 197): Precise GC configured ON
D/dalvikvm( 197): creating instr width table
I/dalvikvm( 197): mlock: /data/dalvik-cache/[email protected]@[email protected], fd=7
So - how do we make the Desire, or any other device, turn on CheckJNI and turn off precise GC?
EDIT: In one of CM's change logs it was written that Precise GC was turned off because of its memory consumption. I am just assuming that this is indeed the cause for my problems. How can we check it?
CheckJNI and Precise GC don't matter.
Repatched and recompiled 2.2 source. The emulator runs beautifully with or without JNI checks. My HTC desire, with a 2.2 rom, still complains about the same function not being native.
The port dudyk did to Omri Baumer's patch works partially - the numbers are not reversed, but the whole line is aligned to the left (with hebrew text appearing to the left of the number, instead of the right).
I have no idea what else to do or to check.
i tried the latest cm6 nightly build and still it won't align Arabic RTL correctly.. it aligns it as LTR and there no reshaping as well....
SOURCE FINDER EXCEL SPREADSHEET
I've been making this table that is specially created for ROM Developers, at first I was doing it for my own knowledge in the rom structure, but why not share? It's not even 50% done (only app/framework so far) but it's worth sharing I think. I will continue to fill in the empty spaces along my building experiences. applying content (more info for table) is greatly appreciated as we all benefit from this table.
Typical scenario where this comes handy: (over dramatized in favor of this spreadsheet)
You're trying to port a rom to your phone, (from source) and you keep getting errors, of course you have logcat as your pet, but still you cannot figure out where to search for that specific thing that keeps crashing, and not to mention in ALL THOSE FOLDER AND FILES (a compiled repo takes upto a [/B]half million[/b] files), of course your developer instincts tell you to do a recursive search but you will find the same folder 20 times, then you think of a recursive grep, but also that returns a ****load of useless info (all links). then you sigh... and take the excel spreadsheet, you lookup the file and browse directly to the right folder, including you know what dependencies it has, so backtracing is easily possible. from here it's smooth sailin' :silly::silly::silly:
Idea's for the "future":
- expand table with binaries from /system/bin, and possibly more folders (after completing libraries)
- make shell script that jumps to the source dir by entering the module/apk/jar name (for example "go services.jar" should jump to frameworks/base/services)
- using "clean services.jar" would delete services_intermediates and it's dependency intermediates. (implemented qc into envsetup.sh, a quick rm -Rf of the chosen intermediates, about 100 times as fast as 'make clean-services for example)
https://github.com/AOSP-S4/android_build/commit/e97fbf5b6c0d52f4b6bbd079659f13ea8c64a22a
Although not very likely, this spreadsheet may contain references that do not apply for your current situation.
Spreadsheet:
https://docs.google.com/spreadsheet/ccc?key=0ArQAWzFXVt6vdGI3anlpNFVTdWtJalUxX2NNTDl5dHc&usp=sharing
PDF is attached
Hi guys,
today I would like to present to you a simple project that was born on this thread to accomplish a very simple task: since there are thousand of Custom ROMs around here, so much of them are LineageOS based, and because of this their forced to build every time all the ROM and post updates here. Users are also forced to check if their preferred ROM was updated or not. That's why I decided to understand how LineageOS OTA Updater System App works. Because of this I've wrote a simple REST Server API emulation that fully works with your ROM (if integrated of course). How? Continue reading down here.
How does it work?
Of course is as simple as it should be. Clone the repo (is a simple PHP website that you can host on any shared hosting*/VPS that you like) and upload it into your preferred hosting. That's it. If you point your web browser to that address the App is already working.
After, you have to upload all your build to the _builds/ folder, and you're done.
The two already working calls /api and /api/v1/build/get_delta should answer correctly the updater app to make it work (remember that visiting it as a browser is not sufficent).
How to integrate it with my ROM?
You have two options:
- Declare cm.updater.uri on your own build.prop file with the value of you own server URL where you have deployed it (This can be done also by the user with any Android App from the Market)
- Replace the string conf_update_server_url_def value inside values.xml of the OTA App source code (COMPILE TIME ONLY!)
Which builds does it support?
Anything that will be borned by the official guide on how to build your custom LineageOS ROM! So, in poor words will be: stable, rcs, nightly and snapshots (likely called EXPERIMENTAL), from CM7 to CM14.
Delta updates SHOULD work too. Just try it and tell me if they works
How can I debug it?
You can use this simple UnitTest that I've already pushed into my GitHub. Feel free to use it everytime you need. It's based upon NodeJS and Unirest.
Is it free?
"Free software is a matter of liberty, not price. To understand the concept, you should think of free as in free speech, not as in free beer."
—Richard Stallman
Click to expand...
Click to collapse
Use it as you want, do anything you want with it as it's MIT licensed.
Is it free of bugs?
That's why I'm here Only you can help me to squash all the remaining bugs!
I hope this will be useful to anyone of you, helping the ROM community providing a simple OTA updater that already works on LineageOS official ROMs.
Greets.
---
Project Home: https://github.com/julianxhokaxhiu/LineageOTA
More about the study: http://blog.julianxhokaxhiu.com/how-the-cm-ota-server-works-and-how-to-implement-and-use-ours
Changelog Build Scripts ( thanks to @Deltadroid ): https://github.com/syphyr/cm_build_scripts/blob/master/make_changelog
* On a Shared Hosting you can ONLY provide a FULL ROM download, NOT DELTAs!
---
Donators:
- @BlueFlame4 x2
Hey, I'm having trouble using your docker image behind a Nginx https reverse proxy. The server is correctly answering the requests:
Code:
{
"id":null,
"response":[
{
"incremental":"",
"api_level":"",
"url":"http:\/\/MYDOMAIN\/\/builds\/full\/lineage-17.1-20200830-UNOFFICIAL-lavender.zip",
"timestamp":1598774045,
"md5sum":"718fb89f935b979edd57b2642234d1fa",
"changes":"",
"channel":"unofficial",
"filename":"lineage-17.1-20200830-UNOFFICIAL-lavender.zip",
"romtype":"unofficial",
"datetime":1598774045,
"version":"17.1",
"id":"50533a894b2ab0d9b2711444ca4f2b530a8ff2389723ea2bd7ada6e029599e2c",
"size":914450521
}
],
"error":null
}
But the returned `url` is http-only and the updater can't download it (throws error). WhenI try to curl it without `-L` I only get `301 Moved Permanently`. Only when I append `-L` I get the correct binary response over https. I'm pretty sure this is a redirecting issue.
Here's my Nginx config:
Code:
server{
server_name MYDOMAIN;
server_tokens off;
listen 80;
listen [::]:80 ipv6only=on;
# Redirect all HTTP requests to HTTPS with a 301 Moved Permanently response.
return 301 https://$host$request_uri;
}
server {
server_name MYDOMAIN;
server_tokens off;
listen 443 ssl http2;
listen [::]:443 ssl http2 ipv6only=on;
location / {
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_set_header X-NginX-Proxy true;
proxy_pass http://127.0.0.1:24087;
proxy_redirect off;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
}
add_header Strict-Transport-Security "max-age=15768000; includeSubdomains; preload";
add_header Referrer-Policy same-origin;
add_header X-Content-Type-Options nosniff;
add_header X-Frame-Options DENY;
add_header X-Xss-Protection "1; mode=block";
[...]
Any idea what I'm doing wrong?
Basically I want to know how to force the `url` response in the JSON to be `https` instead of `http`. (At least that's what I think is the reason for the updater not being able to download the image.)
Code:
08-30 16:00:19.407 7025 7025 D UpdaterController: Starting 50533a894b2ab0d9b2711444ca4f2b530a8ff2389723ea2bd7ada6e029599e2c
08-30 16:00:19.409 7025 7921 E HttpURLConnectionClient: Error downloading file
08-30 16:00:19.409 7025 7921 E HttpURLConnectionClient: java.io.IOException: Cleartext HTTP traffic to MYDOMAIN not permitted
08-30 16:00:19.409 7025 7921 E HttpURLConnectionClient: at com.android.okhttp.HttpHandler$CleartextURLFilter.checkURLPermitted(HttpHandler.java:124)
08-30 16:00:19.409 7025 7921 E HttpURLConnectionClient: at com.android.okhttp.internal.huc.HttpURLConnectionImpl.execute(HttpURLConnectionImpl.java:462)
08-30 16:00:19.409 7025 7921 E HttpURLConnectionClient: at com.android.okhttp.internal.huc.HttpURLConnectionImpl.connect(HttpURLConnectionImpl.java:131)
08-30 16:00:19.409 7025 7921 E HttpURLConnectionClient: at org.lineageos.updater.download.HttpURLConnectionClient$DownloadThread.run(HttpURLConnectionClient.java:250)
08-30 16:00:19.409 7025 7921 E UpdaterController: Download failed
EDIT: Solved!
Code:
location / {
proxy_pass http://127.0.0.1:24087;
proxy_http_version 1.1;
proxy_cache_bypass $http_upgrade;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
#proxy_set_header Host $host;
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-Host $host;
proxy_set_header X-Forwarded-Port $server_port;
proxy_set_header X-NginX-Proxy true;
}
Not sure if this project is still supported, but with build.prop change, it just checks for updates forever on phone. When I try debugging with the UnitTest script, I get "Not Found The requested URL /CyanogenModOTA/api was not found on this server." (definitely server-related) I've gotten this working before on another VPS, so I'm wondering why it's not on my build server.
Are there any specific packages I need to install to get this working?
Here is my php config: http://hongbuild.ddns.net:81/test.php
klvnhng said:
Not sure if this project is still supported, but with build.prop change, it just checks for updates forever on phone. When I try debugging with the UnitTest script, I get "Not Found The requested URL /CyanogenModOTA/api was not found on this server." (definitely server-related) I've gotten this working before on another VPS, so I'm wondering why it's not on my build server.
Are there any specific packages I need to install to get this working?
Here is my php config: http://hongbuild.ddns.net:81/test.php
Click to expand...
Click to collapse
Since build.prop is in the system.new.dat file in lollipop builds, I just started copying it over from my build server and modified the Build.php file to look for the file instead of inside the zip. I just got this working on my web server. Only issue I'm having is related to change logs but I'm going to try and figure that one out later.
Here are the changes I did to get this working with CyanogenMod 12 (all changes will be assumed from the CyanogenModOTA directory):
- mkdir -p builds/buildprop <-- this is where you copy your build.prop from each build (they go in a folder with the same name as the build zip e.x. cm-12-20150103-NIGHTLY)
- mkdir -p builds/changelog <-- this will have your change logs (name them the same as the build zip except with a .txt extension)
- update setConfig( 'basePath', 'CyanogenModOTA' ) to setConfig( 'basePath', 'http://wfhome.net/CyanogenModOTA' ) in index.php (that is my server)
- change the Build constructor function to this (I updated the preg_match_all line, added the buildPropFolder variable, updated the $this->buildProp line, and changed the changeLogUrl line):
Code:
private $buildPropFolder = '';
public function __construct($fileName, $physicalPath) {
/*
$tokens Schema:
array(
1 => [CM VERSION] (ex. 10.1.x, 10.2, 11, etc.)
2 => [DATE OF BUILD] (ex. 20140130)
3 => [CHANNEL OF THE BUILD] (ex. RC, RC2, NIGHTLY, etc.)
4 => [MODEL] (ex. i9100, i9300, etc.)
)
*/
preg_match_all( '/cm-([0-9\.]+-)(\d+-)?([a-zA-Z0-9]+-)?([a-zA-Z0-9]+).zip/', $fileName, $tokens );
$tokens = $this->removeTrailingDashes( $tokens );
$this->filePath = $physicalPath . '/' . $fileName;
$this->buildPropFolder = str_replace('/full', '/buildprop', $physicalPath) . '/' . preg_replace('/\\.[^.\\s]{3,4}$/', '', $fileName);
$this->buildProp = explode( "\n", file_get_contents($this->buildPropFolder . '/build.prop') );
$this->channel = $this->_getChannel( str_replace( range( 0 , 9 ), '', $tokens[3] ) );
$this->filename = $fileName;
$this->url = $this->_getUrl( '', Flight::cfg()->get('buildsPath') );
$this->changelogUrl = str_replace('/full', '/changelog', $this->_getChangelogUrl());
$this->timestamp = filemtime( $this->filePath );
$this->incremental = $this->getBuildPropValue( 'ro.build.version.incremental' );
$this->apiLevel = $this->getBuildPropValue( 'ro.build.version.sdk' );
$this->model = $this->getBuildPropValue( 'ro.cm.device' );
}
Also he has memcached setup in there, so you might make sure you have memcache installed and set to run on startup on your server. You also need xdelta3 to create delta builds although, I don't think it is working ( I compiled the latest version of xdelta3 and it doesn't appear to do anything as there are not any files being created in the delta folder).
Hope that helps.
Thanks for the help (I'll definitely need it when I actually want to start USING the server), but it seems you've misunderstood me. Right now, I can't even get the rest server running properly-that's why I'm getting a 404 error.
Notice you get an output when you go to http://wfhome.net/CyanogenModOTA/api, I don't get anything!
klvnhng said:
Thanks for the help (I'll definitely need it when I actually want to start USING the server), but it seems you've misunderstood me. Right now, I can't even get the rest server running properly-that's why I'm getting a 404 error.
Notice you get an output when you go to http://wfhome.net/CyanogenModOTA/api, I don't get anything!
Click to expand...
Click to collapse
Ah sorry. First thing I see is that you need mod_rewrite installed in apache.
Code:
sudo a2enmod rewrite
sudo service apache2 restart
Need help with development
First I would like to thank you for using this product and playing with it (which involves patching, testing, etc.).
Since the first post I've made here in XDA the project evolved a little since I rewrote entirely the project to make it composer friendly. Rather than that, the Delta build process is actually non working (I tried to create a ZIP but I don't have enough knowledge in ROM development to tell if it's enough or not) so it's just a WIP layer that should be addressed and fixed (I've already found a Python project which does this already but I'm of the idea that this should be somehow not be a bloated software that needs to install binaries here and there to make it working).
So, actually I'm not really working in this project but It's in my TODO list, meanwhile I'll be very happy to have pull requests with useful patches to make this KK, LL and more compatible So feel free to contribute and thanks again for using it!
JulianXhokaxhiu said:
First I would like to thank you for using this product and playing with it (which involves patching, testing, etc.).
Since the first post I've made here in XDA the project evolved a little since I rewrote entirely the project to make it composer friendly. Rather than that, the Delta build process is actually non working (I tried to create a ZIP but I don't have enough knowledge in ROM development to tell if it's enough or not) so it's just a WIP layer that should be addressed and fixed (I've already found a Python project which does this already but I'm of the idea that this should be somehow not be a bloated software that needs to install binaries here and there to make it working).
So, actually I'm not really working in this project but It's in my TODO list, meanwhile I'll be very happy to have pull requests with useful patches to make this KK, LL and more compatible So feel free to contribute and thanks again for using it!
Click to expand...
Click to collapse
Thank YOU for making it! Really appreciate the time and effort you've put into this project.
rjwil1086 said:
Ah sorry. First thing I see is that you need mod_rewrite installed in apache.
Code:
sudo a2enmod rewrite
sudo service apache2 restart
Click to expand...
Click to collapse
That did it, thanks :good:
I've also implemented your changes, but when I check for updates on my phone, I still get "No new updates found". I've copied my new build over to builds/full, and the build.prop to builds/buildprop/cm*
rjwil1086 thank you very much for your help and suggestions, I made my own server http://paksman.ddns.net/cyanogenmodota, edited build.prop for cm updater to look for server url, made all the changes as you but I always get "No new updates found". Not sure if there is a problem with my server or with this project in general. Have you made any success to make this work?
I have it working. I'll upload mine to github tonight
Think I solved my own problem. My builds were all tagged as 'UNOFFICIAL'. When I changed this to 'NIGHTLY' (for the build and build.prop folder respectively ) they finally started to be recognised by cm updater app. JulianXhokaxhiu and rjwil1086, thank you so much for your effort,your work is being much appreciated.
Packsman said:
Think I solved my own problem. My builds were all tagged as 'UNOFFICIAL'. When I changed this to 'NIGHTLY' (for the build and build.prop folder respectively ) they finally started to be recognised by cm updater app. JulianXhokaxhiu and rjwil1086, thank you so much for your effort,your work is being much appreciated.
Click to expand...
Click to collapse
Yup. Sorry. I knew that but forgot to mention it. That's an issue with the CMUpdater app more than it is with the REST implementation. It doesn't look for unofficial files
Packsman said:
Think I solved my own problem. My builds were all tagged as 'UNOFFICIAL'. When I changed this to 'NIGHTLY' (for the build and build.prop folder respectively ) they finally started to be recognised by cm updater app. JulianXhokaxhiu and rjwil1086, thank you so much for your effort,your work is being much appreciated.
Click to expand...
Click to collapse
Awesome! This fixed it for me as well. Thanks to everyone for the help
[CyanogenMod][OTA][PHP] Open Source REST Server for you
Hi, I have been tested server code with a free host (000webhost) then accessed the web, I received messages:
Warning: Unexpected character in input: '\' (ASCII=92) state=1 in /home/a5885282/public_html/index.php on line 27
Parse error: syntax error, unexpected T_STRING, expecting T_CONSTANT_ENCAPSED_STRING or '(' in /home/a5885282/public_html/index.php on line 27
The line 27 of index.php is "use \JX\CmOta\CmOta;". But I don't know about PHP. Please help me solve this issue.
@rjwil1086 , @klvnhng , @Packsman please help me to solve following error:
$ node index.js
<h1>500 Internal Server Error</h1><h3>Undefined offset: 0 (8)</h3><pre>#0 /var/www/CyanogenModOTA/src/Helpers/Build.php(214): flight\Engine->handleError(8, 'Undefined offse...', '/var/www/Cyanog...', 214, Array)
#1 /var/www/CyanogenModOTA/src/Helpers/Build.php(63): JX\CmOta\Helpers\Build->removeTrailingDashes(Array)
#2 /var/www/CyanogenModOTA/src/Helpers/Builds.php(115): JX\CmOta\Helpers\Build->__construct('cm-11-20140103-...', '/var/www/Cyanog...')
#3 /var/www/CyanogenModOTA/src/Helpers/Builds.php(49): JX\CmOta\Helpers\Builds->getBuilds()
#4 /var/www/CyanogenModOTA/vendor/mikecao/flight/flight/core/Loader.php(123): JX\CmOta\Helpers\Builds->__construct()
#5 /var/www/CyanogenModOTA/vendor/mikecao/flight/flight/core/Loader.php(80): flight\core\Loader->newInstance('\JX\CmOta\Helpe...', Array)
#6 /var/www/CyanogenModOTA/vendor/mikecao/flight/flight/Engine.php(69): flight\core\Loader->load('builds', true)
#7 /var/www/CyanogenModOTA/vendor/mikecao/flight/flight/core/Dispatcher.php(191): flight\Engine->__call('builds', Array)
#8 /var/www/CyanogenModOTA/vendor/mikecao/flight/flight/core/Dispatcher.php(191): flight\Engine->builds()
#9 /var/www/CyanogenModOTA/vendor/mikecao/flight/flight/Flight.php(43): flight\core\Dispatcher::invokeMethod(Array, Array)
#10 /var/www/CyanogenModOTA/src/CmOta.php(97): Flight::__callStatic('builds', Array)
#11 /var/www/CyanogenModOTA/src/CmOta.php(97): Flight::builds()
#12 /var/www/CyanogenModOTA/vendor/mikecao/flight/flight/core/Dispatcher.php(160): JX\CmOta\{closure}()
#13 /var/www/CyanogenModOTA/vendor/mikecao/flight/flight/core/Dispatcher.php(143): flight\core\Dispatcher::callFunction(Object(Closure), Array)
#14 /var/www/CyanogenModOTA/vendor/mikecao/flight/flight/Engine.php(310): flight\core\Dispatcher::execute(Object(Closure), Array)
#15 /var/www/CyanogenModOTA/vendor/mikecao/flight/flight/core/Dispatcher.php(191): flight\Engine->_start()
#16 /var/www/CyanogenModOTA/vendor/mikecao/flight/flight/core/Dispatcher.php(142): flight\core\Dispatcher::invokeMethod(Array, Array)
#17 /var/www/CyanogenModOTA/vendor/mikecao/flight/flight/core/Dispatcher.php(48): flight\core\Dispatcher::execute(Array, Array)
#18 /var/www/CyanogenModOTA/vendor/mikecao/flight/flight/Engine.php(64): flight\core\Dispatcher->run('start', Array)
#19 /var/www/CyanogenModOTA/vendor/mikecao/flight/flight/core/Dispatcher.php(191): flight\Engine->__call('start', Array)
#20 /var/www/CyanogenModOTA/vendor/mikecao/flight/flight/core/Dispatcher.php(191): flight\Engine->start()
#21 /var/www/CyanogenModOTA/vendor/mikecao/flight/flight/Flight.php(43): flight\core\Dispatcher::invokeMethod(Array, Array)
#22 /var/www/CyanogenModOTA/src/CmOta.php(80): Flight::__callStatic('start', Array)
#23 /var/www/CyanogenModOTA/src/CmOta.php(80): Flight::start()
#24 /var/www/CyanogenModOTA/index.php(35): JX\CmOta\CmOta->run()
#25 {main}</pre>
Thank you so much.
Thanks all, I fixed.
[CyanogenMod][OTA][PHP] Open Source REST Server for you
I have just creat susscess a server. Thanks.
But, now I want creat a update OTA app for other AOSP ( andoird L) and using that server which I have just creat.
Can I using CMupdater for it? Can you give me some suggest?
Hi, need help to understand!
First of all, i want to say Thank You! For this great job!
I installed server, activated mod rewrite in apache2, apt-get install memcached, made chown for all files to www-data in CyanogenModOTA directory
created directories builds/buildprop, builds/changelog
created file romname.txt in changelog
copied file rom archive to build/full directory
and when i open in browser my ota site: http ota.mydomain.com i see the dir listing as described above
when i try to open url http ota.mydomain.com/api - i see the 404 error.
what exactly i have to do as the next step?
is any json file missed in web root directory (in same place as index.php) ?
and I leave index.php almost unchanged (as in the repository) - just changed string: ->setConfig( 'basePath', '/' )
Thank You once again!
vvzar said:
First of all, i want to say Thank You! For this great job!
I installed server, activated mod rewrite in apache2, apt-get install memcached, made chown for all files to www-data in CyanogenModOTA directory
created directories builds/buildprop, builds/changelog
created file romname.txt in changelog
copied file rom archive to build/full directory
and when i open in browser my ota site: http ota.mydomain.com i see the dir listing as described above
when i try to open url http ota.mydomain.com/api - i see the 404 error.
what exactly i have to do as the next step?
is any json file missed in web root directory (in same place as index.php) ?
and I leave index.php almost unchanged (as in the repository) - just changed string: ->setConfig( 'basePath', '/' )
Thank You once again!
Click to expand...
Click to collapse
Are you sure that modrewrite is working? Can you share a working URL?
lingak said:
I have just creat susscess a server. Thanks.
But, now I want creat a update OTA app for other AOSP ( andoird L) and using that server which I have just creat.
Can I using CMupdater for it? Can you give me some suggest?
Click to expand...
Click to collapse
Honestly I don't know, we have to check if OTA app is the same in Lollipop and works of course the same. If so, we're already safe and yes it can work out of the box. If not, we have to fix it. If you already have a working example, feel free to do a pull request
JulianXhokaxhiu said:
Are you sure that modrewrite is working? Can you share a working URL?
[email protected]:/home/user# a2enmod rewrite
Module rewrite already enabled
[email protected]:/home/user#
.htaccess:
[email protected]:/home/user# cat /var/www/html/CyanogenModOTA/.htaccess
RewriteEngine On
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^(.*)$ index.php [QSA,L]
[email protected]:/home/user#
Is any other rules i have to check?
What about statement <Directory /var/www/html/CyanogenModOTA> ?
Is enouph AllowOverride All and Allow from all ?
test url : http ota.smylink.org
Click to expand...
Click to collapse
vvzar said:
JulianXhokaxhiu said:
Are you sure that modrewrite is working? Can you share a working URL?
[email protected]:/home/user# a2enmod rewrite
Module rewrite already enabled
[email protected]:/home/user#
.htaccess:
[email protected]:/home/user# cat /var/www/html/CyanogenModOTA/.htaccess
RewriteEngine On
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^(.*)$ index.php [QSA,L]
[email protected]:/home/user#
Is any other rules i have to check?
What about statement <Directory /var/www/html/CyanogenModOTA> ?
Is enouph AllowOverride All and Allow from all ?
test url : http ota.smylink.org
Click to expand...
Click to collapse
Technically it should be enough but the order of where you placed it is important too.
Anyway, going to your URL is just making me think that the PHP code is running well (the redirect to builds folder is triggered by CMOTA Rest Code). You're just missing the htaccess rules.
I'm quite sure you just have to figure out your own server setup to understand if mod_rewrite is properly working (allowing htaccess files to be read and parsed).
Click to expand...
Click to collapse