Related
Optware was originally developed as a distribution mechanism for software for the Open Source Linux project for the Linksys NSLU2 (a small low cost network storage device). That consumer storage device ran on Linux, and the community opened it up, but they needed a lightweight distribution mechanism for their lightweight devices.
Since 2004 Optware has been adopted by a variety of Linux communities and device developers including the webOS community working on the Palm Pre and Pixi, and developers for devices as diverse as Asus WL-500g,WL-HDD, WL-500gx, WL-500gP routers, the Synology DS-101 and DS-101g+ NAS devices.
At its core, Optware is a package manager to allow you to easly install and uninstall cross-compiled linux applications on your device over the internet. At this time, optware supports the automatic download and installation of over 1500 programs. (See the list at the end of the wiki article.)
Why should I use Optware?
Like the original slug, android devices lack a distribution mechanism to install Linux/command line tools programs and utilities. Many in the Android community who want or need to do command line processing on their device use a chroot of some sort (Debian seems popular) to get their distribution and package management tools. With Optware, it becomes possible to simply run whatever utility or program you want on the native underlying Linux.
Additionally, Optware For Android installs openSSH making it possible to access your Nook from any PC with SSH capability, and sets up an unprivileged username/password login system for the device at the command line. All in all then, since it can be installed with a single script, since it's lightweight, and since it's designed for small and embedded devices, Optware is a perfect fit for the Android community.
Optware for Nook is the first Optware for Android release by the Novaports team and is their gift to the Android community members who have helped them so much with information on Android devices.
What does it do?
The Optware bootstrap installed the IPKG package manager. It installs the optware Busybox. It installs openSSH and makes the ssh demon persistent. It creates an unprivileged user with a username and password you choose.
See Installation instructions and details at the Optware for Android page at Nookdevs.com
For completeness of licensing compliance, the source code for all optware package can be found at the following locations:
1) build scripts at svn.nslu2-linux.org in the /svnroot/optware/ area
2) tarballs at sources.nslu2-linux.org in the /sources area (this is a *large* directory)
To contribute code to Optware, see www . nslu2-linux . org in the /wiki/Optware/AddAPackageToOptware area
-- Rod Whitby
-- Original NSLU2-Linux Project Lead and Optware Architect
This works beautifully! Thank you!
Gonna try this and see if i can get samba to work.. a SMB mount over wifi would be awsome...
Many thanks!
I don't suppose there's a quick and easy way to completely remove optware?... not looking forward to culling through scripts and manually un-doing it all.....
adduser: not found
When it gets to the part in the install script to add the username of the unprivileged user, I enter a name, then it says: adduser: not found.
If it helps, I am using Rooted Stock 1.1.
Anyone get Optware to install on CM7 (newer nightly)
I am running CM7 #103 and would love to have optware installed.
But I get Fail if I try to use the method from Nookdevs
Tonight when I have more time I will look at the errors and see if I can tell what is up.
It looked like partially due to missing dir's for one (/tmp)
madrascafe said:
Gonna try this and see if i can get samba to work.. a SMB mount over wifi would be awsome...
Click to expand...
Click to collapse
The market has your answer and it's free:
https://market.android.com/details?id=com.funkyfresh.samba&feature=search_result
Am I the only one that wants Optware on CM7?
It goes fine on a manualnootered 1.2.0 NC But I prefer CM7 (And use the nightlies)
I would love to get this going on my NC as well. Running CM7 N121 w/ 6/30/2011 OC kernel. Was able to manually work past the tmp directory problem you mentioned, but I can't seem to execute the ipkg binary or installed. I've verified it indeed exists in the path I'm trying to execute it from, it has +x perms for user/ group/ other, but it always gives the error below:
/data/opt/bin/ipkg: not found
I've tried messing with bin directory and ipkg binary ownership, but no dice. If anything it should at least find it even if permissions aren't correct.
Looks like the contents of the binary reference ELF libraries which sounds like an issue attempting to execute in an Android environment. Going to see if I can chroot an environment that will work for it.
Sent from my NookColor using Tapatalk
This is happening because it's looking for /lib/ld-linux.so.3, and not finding it. I worked around by
ln -s /data/opt/lib/ld-linux.so.3 /lib/ld-linux.so.3
I also had to make these links:
ln -s /data/opt /opt
ln -s /data/tmp /tmp
And then set this:
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/lib
export PATH=$PATH:/opt/bin/:/opt/sbin
in my bash profile to get things to start working.
One problem is my symlinks seem to get erased every reboot. I worked around by re-creating them in my bash profile, but I wish I knew how to make them permanent. This may be a ROM specific thing - this was written for a rooted stock nook, and I'm on an Epic running SFR.
I do not have a ld-linux.so.3 file on my NC w/ CM7 125
I have many other errors that show up:
Downloading the latest ipkg-opt package from the Optware package feed:
Connecting to ipkg.nslu2-linux.org (140.211.169.161:80)
wget: can't open 'ipkg-opt_0.99.163-10_arm.ipk': File exists
(NOTE the above file 'ipkg-opt_0.99.163-10_arm.ipk' does not exist
(on my NC - from a find / -name ipkg-opt_0.99.163-10_arm.ipk)
Updating the Optware package database:
ERROR: Failed to update the local Optware package database
Updating the Optware package database:
/data/optware-bootstrap-0.0.1.shar: ipkg-opt: not found
ERROR: Failed to update the local Optware package database
Any ideas?
The actual file it needs is /data/opt/lib/ld-linux.so.3 - you need to create a symlink in /lib in order to see it. But I think that file comes from the optware installer - if the installer is failing, you won't have it there either. It sounds like you may have bits of a previous install that failed, and it's preventing the install from running properly. Try looking at the log in /data/tmp for more info. After that, try rm -rf /data/tmp/* to clean up the old install and trying again.
Thank You for any pointers.
I deleted everything in /data/tmp first last attempt - like you suggested.
Same result
I have it down to only this error now when I run it (optware-bootstrap-0.0.1.shar):
Updating the Optware package database:
optware-bootstrap-0.0.1.shar: ipkg-opt: not found
ERROR: Failed to update the local Optware package database
ipkg-opt should be in /data/opt/bin. Check to see if it's there... if not, something went wrong in one of the previous steps - probably somewhere around "Downloading the latest ipkg-opt package from the Optware package feed". See if wget is actually downloading anything, where it's putting it, and what it's doing with it afterwards.
If it is there, try running it manually. If you see /data/opt/bin/ipkg or /data/opt/bin/ipkg-opt but it claims it's not there when you run it, it may be a library linking error. Mine was looking for /lib/ld-linux.so.3 but it was actually installed at /data/opt/lib/ld-linux.so.3. mount -o remount,rw followed by ln -s /data/opt/lib/ld-linux.so.3 /lib/ld-linux.so.3 fixed that. I also had to run
ln -s /data/opt /opt
ln -s /data/tmp /tmp
before things started working.
By the way:
ntfsprogs - NTFS filesystem libraries and utilities
Could that be of any use? At the moment, it's not possible to mount NTFS storage on the NC
i would love to know how you guys get to this point even. When i try to install the very fist line of the script is an error and then nothing at all happens .... something about not finding busybox or something of that nature.
So you guys are getting WAY farther than I've been able to.
Tkx Tonygoes.
It is there in /data/opt/bin but it reports /data/opt/bin/ipkg: not found
I cannot find a ld-linux.so.3 on my NC at all.
I did the other links et al fro your comments also
obsid:
Try post 12 above:
and I had some other suggestions from someone:
might try thses FIRST the script seems to do it but may not actually get it done.
mount -o remount,rw / /
mount -o remount,rw /dev/block/mmcblk0p5 /system
you should also if tried before do this: rm -rf /data/tmp/
then do the items in post #12 above
NOTE for me the ln -s to ld-linux.so.3 does not work as I do not have that file on my system
Hello friends! I post this on the off to pic because I can't really find a fitting sub-forum. If you know a better place I can post this thread, please let me know.
I want ruby and Ruby gems to run on my android device (galaxy S4 GT-I9515, not that it matters). I know android is Linux based so there shouldn't be a problem doing this.
Basically, I don't want to use Ruby for development, I just need to use some of it's applications (gems).
I tried ruboto core and ruboto IRB, but couldn't install a gem.
In general, my android won't read "apt-get" command in terminal. I don't know why.
If anyone could help me to run some Ruby gems on my device, I would really appreciate it. Thanks for people who answer!
Android is a Linux kernel, however it is not a full Linux like Debian or Ubuntu.
Android doesn't have a package manager by default (unless with Google Play or other app markets). It doesn't have apt-get.
However, I can say that there's a terminal called Termux in Google Play or F-Droid.
It does have apt and it doesn't require root.
I found Ruby in the apt repos of Termux though.
Code:
$ apt list | grep ruby
WARNING: apt does not have a stable CLI interface. Use with caution in scripts.
ruby/stable 2.3.3 arm
ruby-dev/stable 2.3.3 arm
ruby-ri/stable 2.3.3 all
weechat-ruby-plugin/stable 1.6-1 arm
TheKindleMCPEGuy said:
Android is a Linux kernel, however it is not a full Linux like Debian or Ubuntu.
Android doesn't have a package manager by default (unless with Google Play or other app markets). It doesn't have apt-get.
However, I can say that there's a terminal called Termux in Google Play or F-Droid.
It does have apt and it doesn't require root.
I found Ruby in the apt repos of Termux though.
Code:
$ apt list | grep ruby
WARNING: apt does not have a stable CLI interface. Use with caution in scripts.
ruby/stable 2.3.3 arm
ruby-dev/stable 2.3.3 arm
ruby-ri/stable 2.3.3 all
weechat-ruby-plugin/stable 1.6-1 arm
Click to expand...
Click to collapse
First of all thanks for the reply!
So, in order to use ruby on android all I have to do is use a more linux-like terminal, and use the command "$ apt list | grep ruby", am I right?
Will I have access to apt-get and commands like that with the new terminal?
FurySh0ck said:
Android is a Linux kernel, however it is not a full Linux like Debian or Ubuntu.
Android doesn't have a package manager by default (unless with Google Play or other app markets). It doesn't have apt-get.
However, I can say that there's a terminal called Termux in Google Play or F-Droid.
It does have apt and it doesn't require root.
I found Ruby in the apt repos of Termux though.
First of all thanks for the reply!
So, in order to use ruby on android all I have to do is use a more linux-like terminal, and use the command "$ apt list | grep ruby", am I right?
Will I have access to apt-get and commands like that with the new terminal?
Click to expand...
Click to collapse
Termux is not really more Linux like, it's still using the same kernel. It downloads a core (other applications, etc) to /data folder. That's how it works.
You will have access to apt in the new terminal. However, there is not much packages in the repo as your Android device is ARM and the packages have to be built into a package file. If you'd like the full experience, consider using a KVM or Xen VPS which run full on distributions (Debian, Ubuntu, etc). They also have more packages over there.
Now to install Ruby, you wouldn't apt list. What you would do in Termux:
Code:
apt install ruby
would install Ruby and you would be able to use it and install gems.
I installed Ruby with that command and confirmed it works:
Code:
$ apt install ruby
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following NEW packages will be installed:
ruby
0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
Need to get 3587 kB of archives.
After this operation, 18.3 MB of additional disk space will be used.
Get:1 [url]http://termux.net[/url] stable/main arm ruby arm 2.3.3 [3587 kB]
Fetched 3587 kB in 1s (2878 kB/s)
Selecting previously unselected package ruby.
(Reading database ... 2137 files and directories currently installed.)
Preparing to unpack .../archives/ruby_2.3.3_arm.deb ...
Unpacking ruby (2.3.3) ...
Processing triggers for man (1.13.4-3) ...
Setting up ruby (2.3.3) ...
When I ran the
Code:
gem
command I got this:
Code:
$ gem
RubyGems is a sophisticated package manager for Ruby. This is a
basic help message containing pointers to more information.
Usage:
gem -h/--help
gem -v/--version
gem command [arguments...] [options...]
Examples:
gem install rake
gem list --local
gem build package.gemspec
gem help install
Further help:
gem help commands list all 'gem' commands
gem help examples show some examples of usage
gem help gem_dependencies gem dependencies file guide
gem help platforms gem platforms guide
gem help <COMMAND> show help on COMMAND
(e.g. 'gem help install')
gem server present a web page at
[url]http://localhost:8808/[/url]
with info about installed gems
Further information:
[url]http://guides.rubygems.org[/url]
Enjoy!
TheKindleMCPEGuy said:
Termux is not really more Linux like, it's still using the same kernel. It downloads a core (other applications, etc) to /data folder. That's how it works.
You will have access to apt in the new terminal. However, there is not much packages in the repo as your Android device is ARM and the packages have to be built into a package file. If you'd like the full experience, consider using a KVM or Xen VPS which run full on distributions (Debian, Ubuntu, etc). They also have more packages over there.
Now to install Ruby, you wouldn't apt list. What you would do in Termux:
Code:
apt install ruby
would install Ruby and you would be able to use it and install gems.
I installed Ruby with that command and confirmed it works:
Code:
$ apt install ruby
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following NEW packages will be installed:
ruby
0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
Need to get 3587 kB of archives.
After this operation, 18.3 MB of additional disk space will be used.
Get:1 [url]http://termux.net[/url] stable/main arm ruby arm 2.3.3 [3587 kB]
Fetched 3587 kB in 1s (2878 kB/s)
Selecting previously unselected package ruby.
(Reading database ... 2137 files and directories currently installed.)
Preparing to unpack .../archives/ruby_2.3.3_arm.deb ...
Unpacking ruby (2.3.3) ...
Processing triggers for man (1.13.4-3) ...
Setting up ruby (2.3.3) ...
When I ran the
Code:
gem
command I got this:
Code:
$ gem
RubyGems is a sophisticated package manager for Ruby. This is a
basic help message containing pointers to more information.
Usage:
gem -h/--help
gem -v/--version
gem command [arguments...] [options...]
Examples:
gem install rake
gem list --local
gem build package.gemspec
gem help install
Further help:
gem help commands list all 'gem' commands
gem help examples show some examples of usage
gem help gem_dependencies gem dependencies file guide
gem help platforms gem platforms guide
gem help <COMMAND> show help on COMMAND
(e.g. 'gem help install')
gem server present a web page at
[url]http://localhost:8808/[/url]
with info about installed gems
Further information:
[url]http://guides.rubygems.org[/url]
Enjoy!
Click to expand...
Click to collapse
Awesome! It worked well and neat. +1 For your Thanks-o'-meter.
Now I have some questions about termux, if you could answer:
1) Where does it save all the files? I used
Code:
apt update
and I am going to download packages from now on. I think I should know where it goes / how can I change that save directory.
2) When I tried using root permissions (you know, my device is rooted so I used
Code:
su
in order to gain access) I couldn't use apt commands anymore. I think some of my ruby gems will require root permissions, so I would like further explanation about it (only if you could of course )
FurySh0ck said:
Termux is not really more Linux like, it's still using the same kernel. It downloads a core (other applications, etc) to /data folder. That's how it works.
You will have access to apt in the new terminal. However, there is not much packages in the repo as your Android device is ARM and the packages have to be built into a package file. If you'd like the full experience, consider using a KVM or Xen VPS which run full on distributions (Debian, Ubuntu, etc). They also have more packages over there.
Now to install Ruby, you wouldn't apt list. What you would do in Termux:
would install Ruby and you would be able to use it and install gems.
I installed Ruby with that command and confirmed it works:
When I ran the command I got this:
Awesome! It worked well and neat. +1 For your Thanks-o'-meter.
Now I have some questions about termux, if you could answer:
1) Where does it save all the files? I used and I am going to download packages from now on. I think I should know where it goes / how can I change that save directory.
2) When I tried using root permissions (you know, my device is rooted so I used in order to gain access) I couldn't use apt commands anymore. I think some of my ruby gems will require root permissions, so I would like further explanation about it (only if you could of course )
Click to expand...
Click to collapse
1. It saves inside app data, specific folder is /data/data/com.termux/files/usr/var/cache/apt
2. I don't think su works in Termux really, however my root doesn't work right now so I can't really say a definite answer.
TheKindleMCPEGuy said:
1. It saves inside app data, specific folder is /data/data/com.termux/files/usr/var/cache/apt
2. I don't think su works in Termux really, however my root doesn't work right now so I can't really say a definite answer.
Click to expand...
Click to collapse
That's all I had to ask in this thread.
Thanks!
Hello friends!
I am using Termux in order to run ruby.
I installed ruby successfully with "apt" command, and it functions fine.
Current ruby version:
Code:
$ ruby -v
ruby 2.3.3p222 (2016-11-21 revision 56859) [arm-linux-androideabi]
The problem is, I can't really install gems. I tried to install bettercap for the experiment, but it failed. This is what I get:
Code:
$ gem install bettercap
Fetching: colorize-0.8.1.gem (100%)
Successfully installed colorize-0.8.1
Fetching: network_interface-0.0.1.gem (100%)
Building native extensions. This could take a while...
ERROR: Error installing bettercap:
ERROR: Failed to build gem native extension.
current directory: /data/data/com.termux/files/usr/lib/ruby/gems/2.3.0/gems/network_interface-0.0.1/ext/network_interface_ext
/data/data/com.termux/files/usr/bin/ruby -r ./siteconf20161129-14856-1cclchu.rb extconf.rb
mkmf.rb can't find header files for ruby at /data/data/com.termux/files/usr/lib/ruby/include/ruby.h
extconf failed, exit code 1
Gem files will remain installed in /data/data/com.termux/files/usr/lib/ruby/gems/2.3.0/gems/network_interface-0.0.1 for inspection.
Results logged to /data/data/com.termux/files/usr/lib/ruby/gems/2.3.0/extensions/arm-linux/2.3.0/network_interface-0.0.1/gem_make.out
I found some instructions if I ran "gem help install" but I couldn't really understand what to do in order to fix that.
Can anyone help me to solve this? BTW I have a rooted device so I can use "su" and "sudo" and all that stuff...
Thanks for people who answer~
FurySh0ck said:
Code:
mkmf.rb can't find header files for ruby at /data/data/com.termux/files/usr/lib/ruby/include/ruby.h
Click to expand...
Click to collapse
You can install ruby.h with:
Code:
apt install ruby-dev
.
fornwall said:
You can install ruby.h with: .
Click to expand...
Click to collapse
I installed ruby-dev but it still won't work. It tells me it saved a log file which contains the explanation to the failure. I'll post the whole code, but please pay attention to the last part of it:
Code:
apt install bettercap
Reading package lists... Done
Building dependency tree
Reading state information... Done
E: Unable to locate package bettercap
$ gem install bettercap
Building native extensions. This could take a while...
ERROR: Error installing bettercap:
ERROR: Failed to build gem native extension.
current directory: /data/data/com.termux/files/usr/lib/ruby/gems/2.3.0/gems/network_interface-0.0.1/ext/network_interface_ext
/data/data/com.termux/files/usr/bin/ruby -r ./siteconf20161206-31345-1xerjr9.rb extconf.rb
[*] Running checks for netifaces code...
[*] Warning : this platform as not been tested
checking for getifaddrs()... *** extconf.rb failed ***
Could not create Makefile due to some reason, probably lack of necessary
libraries and/or headers. Check the mkmf.log file for more details. You may
need configuration options.
Provided configuration options:
--with-opt-dir
--without-opt-dir
--with-opt-include
--without-opt-include=${opt-dir}/include
--with-opt-lib
--without-opt-lib=${opt-dir}/lib
--with-make-prog
--without-make-prog
--srcdir=.
--curdir
--ruby=/data/data/com.termux/files/usr/bin/$(RUBY_BASE_NAME)
/data/data/com.termux/files/usr/lib/ruby/2.3.0/mkmf.rb:456:in `try_do': The compiler failed to generate an executable file. (RuntimeError)
You have to install development tools first.
from /data/data/com.termux/files/usr/lib/ruby/2.3.0/mkmf.rb:541:in `try_link0'
from /data/data/com.termux/files/usr/lib/ruby/2.3.0/mkmf.rb:556:in `try_link'
from /data/data/com.termux/files/usr/lib/ruby/2.3.0/mkmf.rb:765:in `try_func'
from /data/data/com.termux/files/usr/lib/ruby/2.3.0/mkmf.rb:1051:in `block in have_func'
from /data/data/com.termux/files/usr/lib/ruby/2.3.0/mkmf.rb:942:in `block in checking_for'
from /data/data/com.termux/files/usr/lib/ruby/2.3.0/mkmf.rb:350:in `block (2 levels) in postpone'
from /data/data/com.termux/files/usr/lib/ruby/2.3.0/mkmf.rb:320:in `open'
from /data/data/com.termux/files/usr/lib/ruby/2.3.0/mkmf.rb:350:in `block in postpone'
from /data/data/com.termux/files/usr/lib/ruby/2.3.0/mkmf.rb:320:in `open'
from /data/data/com.termux/files/usr/lib/ruby/2.3.0/mkmf.rb:346:in `postpone'
from /data/data/com.termux/files/usr/lib/ruby/2.3.0/mkmf.rb:941:in `checking_for'
from /data/data/com.termux/files/usr/lib/ruby/2.3.0/mkmf.rb:1050:in `have_func'
from extconf.rb:43:in `<main>'
To see why this extension failed to compile, please check the mkmf.log which can be found here:
/data/data/com.termux/files/usr/lib/ruby/gems/2.3.0/extensions/arm-linux/2.3.0/network_interface-0.0.1/mkmf.log
extconf failed, exit code 1
Gem files will remain installed in /data/data/com.termux/files/usr/lib/ruby/gems/2.3.0/gems/network_interface-0.0.1 for inspection.
Results logged to /data/data/com.termux/files/usr/lib/ruby/gems/2.3.0/extensions/arm-linux/2.3.0/network_interface-0.0.1/gem_make.out
Any solutions in mind?
BTW Thanks for your time, I appreciate anyone who tries to help.
Hi,
before Oreo I happily used "Rsync for Android" to sync images from my NAS to my Pixel C (and backups the other way).
Since Oreo this no longer works. The rsync and ssh executables in /sdcard/Android/data/eu.kowalczuk.rsync4android/files/ are no longer executable, as the sdcard is mounted noexec.
The devices is rooted, stock ROM.
I am looking for either alternative apps that do not rely on executables on the sdcard or ways to make "Rsync for Android" usable again. Preferable a solution that survives system updates.
thx
afx
I am on the same boat. Any ideas?
prescaler said:
I am on the same boat. Any ideas?
Click to expand...
Click to collapse
Ok,
found something today:
Rsync Wrapper by Letscorp from the play store.
seems to work, though one needs to experiment a bit with the options.
cheers
afx
Rsync Wrapper doesn't work for me even on Nougat. One review says it may be a problem with running it on Lineage.
Could not create directory '/data/.ssh'.
Host key verification failed.
rsync: connection unexpectedly closed (0 bytes received so far) [sender]
rsync error: error in rsync protocol data stream (code 12) at external/rsync/io.c(226) [sender=3.1.1]
I used llama + rsync4android to backup DCIM/Camera to my Linux server. Same problem w/ Oreo. The easiest work around is to use SmbSync2 instead. The other workaround is to use root the phone and use one of the following:
- llamaLab's Automate. (Search the community contributed tasks for an rsync example).
- Tasker + com.termux's plugin (I haven't tested this one)
The really hacker's method (My first attempt):
- install rsync4android
- install the rsync binaries (rsync, ssh, dropbearkey)
- generate dss_key
- as root, copy those binaries to to /system/bin (will need remount /system rw)
- chmod 755 those binaries
- install a cron scheduler (I used ru.kslabs.scheduler)
- create a task and directly run rsync. (Might need to run as root)
I have the same issues with rsync4android on LineageOS 15.1 (Oreo 8.1). The "really hacker's method" do not work at all. With my old, yet broken cellphone, running with CarbonROM Oreo 8.1, rsync4android worked fine.
Any ideas?
Any alternatives?
Been using rsync wrapper for quite a wile now and it works just fine (two stock Oreo systems).
Give it a try.
cheers
afx
a.f.x. said:
Been using rsync wrapper for quite a wile now and it works just fine (two stock Oreo systems).
Give it a try.
Click to expand...
Click to collapse
Thank you. I tried again with various options and for hours.
I always get this:
rsync: connection unexpectedly closed 90 bytes received so far) [Receiver]
rsync error: error in rsync protocol data stream (code 12) at external/rsync/io.c(226) [Receiver=3.1.2]
I found a solution for this issue.:
Connect the smartphone by USB to a computer.
Mount it with JMTPFS.
Write a bash script with different rsync commands.
Execute the rsync bash script.
That will do it for me. For Calendar and Contacts synchronization I use Radicale.
Still, the SD Card remains unencrypted and Adoptable SD Cards are no longer possible. I blame Google for this retrograde and insecure step.
Sorry, I was wrong. The mount with jmtpfs + rsync method did not work. Unfortunately, the files were not copied to the phone. For me this Google-FBE is a big disadvantage for Android and I have a broken smartphone, I want to get rid of it.
I faced 2 issues with rsync4android:
- The binaries didn't seem to work with Oreo
- Dropbearkey generates a DSS key instead of a RSA one
So I installed rsync wrapper and with the key it generates, it worked. But I want to be able to automate my backups with Tasker so it wbsn't perfect. So I overwrote the binaries of /data/data/eu.kowalczuk.rsync4android/files with those from /data/data/net.letscorp.rsyncwrapper/files. with the RSA key, it did work with rsync4android and Tasker
I finally got rsync wrapper to work in Lineage 15.1 by converting my dropbear formatted key to openssh format, and replacing my old rsync -e option with...
-e "ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -p 22 -i '/sdcard/id_rsa'"
Running LineageOS 15.1 and here is my finding:
The issue is with ssh client included in ROM. Workaround is to update the -e parameter in to include full path to local ssh instead
From: -e "ssh -l <your username>...
To: -e "/data/data/net.letscorp.rsyncwrapper/files/ssh -l <your username>...
An easy fix for Rsync Wrapper. But I was unable to edit command line for Rsync4android.
I just cannot get Rsync Wrapper to work, even playing with the options. My rsync4android command was
Code:
rsync -vHrltD --chmod=Du+rwx,go-rwx,Fu+rw,go-rw --no-perms --exclude=.trashcan --exclude=.android_secure --delete-after -e "ssh -y -p 22 -i '/sdcard/dss_key'" /storage/F975-D0D2/ [email protected]:/media/HDD/folder
I tried a number of variations including the ones you all suggested, with the end string being
Code:
rsync -vHrltD --chmod=Du+rwx,go-rwx,Fu+rw,go-rw --no-perms --exclude=.trashcan --exclude=.android_secure --delete-after -e "/data/data/net.letscorp.rsyncwrapper/files/ssh -y -p 22 -i '/sdcard/dss_key" /storage/F975-D0D2/ [email protected]:/media/HDD/folder
It just does not work: I always get as a result
Code:
/data/data/net.letscorp.rsyncwrapper/files/ssh: Exited: Error connecting: Connection refused
rsync: connection unexpectedly closed (0bytes received so far) [sender]
rsync error: error in rsync protocol data stream (code 12) at io.c(226) [sender=3.1.2]
Would you kindly suggest any other attempts I could try? My phone is not rooted, and I wish to backup my SD Card to a raspberrypi I have in my home; with Nougat it worked wonderfully and it still does from my girlfriend's Nougat phone.
So, I have been an Android developer since 2009 (HTC dream), and have been actively "consuming" XDA, custom ROMs and other tweaks. Surprisingly, never tried to build my own ROM from scratch.
Recently, something made me want to build the ROM, so that I can make some changes to the way SystemUI behaves (specifically putting some app shortcuts to my Pixel phone(s), like camera etc. which are now removed in Android 10). So, after a week's struggle I got to where I wanted to reach. (90% time spent in getting the first successful flash. 1% feature development. 9% feature polishing).
Here is my guide to all beginners. (It is pretty simple, if you know the steps).
System setup
I have always been a Windows user (and I love my Surface(s)), but you cannot build Android on Windows machines (as clearly called out in source.android.com). I still tried to install Ubuntu shell from Microsoft store, and build (Spoiler alert: Does not work).
Next is Mac. Android can be built in Mac, I got it build in Mac. But, it is not easy. Especially with setting up the environment, having the right version of MacOS (doesn't work on Catalina yet). And also, challenges with filesystem format (Android building only works on case sensitive file system, so you have to create such a partition). Android building needs at least 160GB of disk space (so unless you are super rich and have 512GB+ Macbook with top specs, it is going to be hard).
My choice machine hence became, my two desktops (i7 4 core, 16GB, 1TB SSD, Ubuntu 18.04 and Xeon 12 core, 32GB, 512GB disk, with Ubuntu 18.04).
There is a reason why I specifically talk about these two machines. To build Android fast (cold clean build in less than 4 hours), you need
Fast processors, and more cores
Lots of RAM
A SSD disk (with 200GB space)
If you are missing any of the above 3, you will build times will go up. I have found for hot build, both machines did a decent job (2-3 mins if you are working on single module), but SSD was more important than cores, and RAM.
Setting up your Ubuntu machine. {ETA 30 mins}
Android has official (and clearly laid out) steps here.
But for Ubuntu these are pretty much the steps.
Code:
$sudo apt-get install git-core gnupg flex bison gperf build-essential zip curl zlib1g-dev gcc-multilib g++-multilib libc6-dev-i386 lib32ncurses5-dev x11proto-core-dev libx11-dev lib32z-dev libgl1-mesa-dev libxml2-utils xsltproc unzip
And the guide doesn't mention this, but you need Python2.7, so get this.
Code:
# refreshing the repositories
sudo apt update
# its wise to keep the system up to date!
# you can skip the following line if you not
# want to update all your software
sudo apt upgrade
# installing python 2.7 and pip for it
sudo apt install python2.7 python-pip
# installing python-pip for 3.6
sudo apt install python3-pip
Also install adb.
Code:
sudo apt install android-tools-adb android-tools-fastboot
If you have come till here, you're ready to build for different devices.
Getting the code ready to build {ETA 5 hours - 1 day}
Most of this is also mentioned in the AOSP official website, but some stuff are tricky, I will try to highlight those steps here.
We are going to build the ROM for Pixel 3 (Android 10 - QP1A.191105.003 )
Download and explode the code {ETA 2-3 hours, depending on internet speed}
Here we are talking about downloading at least 20GB of code (text heavy content) over the internet. Going to be excruciatingly slow.
Also, we will be downloading code for specific device model, so if you want to do it for a newer model, you will have to go through the grind again.
Although, technically it might be possible to have the same folder contain code for multiple devices, it is too risky IMO, something goes wrong, you lose everything.
Recommended folder structure would be
aosp --> device 1
aosp --> device 2
......
aosp --> device n
Click to expand...
Click to collapse
With each folder containing over 150GB of contents (after downloading, building etc), so in practical sense, n could be only 3-4 at max.
Setting up repo.
Repo is a tool that Google uses to checkout and manage the AOSP code in your local machine. Once you download the codebase, you can use the command to resync, update, code base.
Code:
mkdir ~/bin
PATH=~/bin:$PATH
You should persist this folder in your PATH variable all the times.
Code:
curl https://storage.googleapis.com/git-repo-downloads/repo > ~/bin/repo
chmod a+x ~/bin/repo
This sets up repo in your machine.
One final step before you actually start the long download, setup your git details.
Code:
git config --global user.name "Your Name"
git config --global user.email "[email protected]"
Now download the code. Like I previously suggested I would do this.
Code:
mkdir ~/aosp
cd ~/aosp
mkdir pixel3
cd pixel3
Now, let's start getting the code home.
Code:
repo init -u https://android.googlesource.com/platform/manifest -b android-10.0.0_r10 --depth=1
Here we have done a bunch of things.
We have chosen a particular Android build tag to download (branch). You can follow the link to choose which branch you want to checkout, based on your test phone and Android version you want to build.
We have asked to only download the latest version of the branch and not all of the branch (--depth), this considerably reduces our download time.
Now that we have decided what to download, let's download the code with this command.
Code:
repo sync -qc -j4
This command is going to take a while to download over 20GB of code. In the meanwhile, let's see what we did here.
-q Asks the download to be silent (which means it will show just overall progress)
-c Makes sure we are only downloading current branch
-j[x] This the tricky one. Let's talk about this.
With -j we are asking repo to spawn multiple downloads (parallelly), to speed up the process. We will see this flag going forward in other places also. We should keep the value of x to number of cores we have in our machine. To find how many cores you have run
Code:
nproc --all
. Note that I have had situations where I put a very high value for n (higher than my cores as well), and eventually ran my JVM out of RAM to run the command (in parallel). So, the trade off here is to restrict it to the core number.
***Key step: Download radio drivers.***
Most tutorials miss this or mention it very subtly. But, without this step the ROM you flash won't boot to the home screen (you will be in the boot loop).
Go to the driver binaries page, and download the right zip files for the Android build version (android-10.0.0_r10) and device (Pixel 3) you chose earlier in the repo command.
You will be downloading two zip files (one vendor image zip and one radio drivers zip), both zips will have on shell script file each (.sh), just put those two files in your repo folder (~aosp/pixel3) and run the scripts. It will download the required proprietary files (after asking you to accept the terms). Do not miss this step.. I lost 3 days trying to find the reason for my ROM not booting up, this was it.
Let's build our code
Now things are more definitive.
Code:
source build/envsetup.sh
This command basically sets up your environment, adding necessary commands to path etc.
Code:
lunch aosp_blueline-userdebug
You can read more about this command here.
Basically this sets up the right parameters to build for your specific model. The param can derived based on aosp_[device code]-[userdebug | eng | user].
Once you have run the above two commands, you can *finally* build your codebase.
Code:
m droid -j4
m basically makes and builds the whole codebase.
Code:
droid
refers to the defaults target configuration (optional). -jN is to specify parallelism (equal to number of cores you have).
This command could take anywhere between 4-12 hours for the first run. But, if you followed all steps above, you should have a green message in the end saying this
Code:
[COLOR="SeaGreen"]#### build completed successfully (2:03:04 (hh:mm:ss)) ####[/COLOR]
Flashing your phone
Now, you're 50% safe when your build has finished successfully. Now, next 50% depends if you're able to flash it and get the phone booting.
This part most of you should know, so I am keeping it brief.
Enter fastboot
Code:
adb reboot bootloader
Unlock your bootloader
Code:
fastboot flashing unlock
Flash your Build
From the root folder of your repository (~/aosp/pixel3)
Code:
fastboot flashall -w
In a few minutes your device should be booting to the freshing baked ROM that you made.
What next?
You can just repeat
Code:
m droid -j4
to repeat builds
You can also go to a specific module folder and execute
Code:
mm
to only build that module
You can use adb sync to update specific modules without flashing again (this never worked for me, always bricked my device)
Use *fastboot flashall* without [-w] flag to flash over existing ROM without losing user data.
You can clean up the whole builds and rebuild everything from scratch. Run
Code:
make clobber
to clean your build, and use
Code:
m
to build again
You could face adb issues (device not detected) in Ubuntu. I am not going into details of how to fix that
This has been pretty much my journey so far with AOSP. I am comfortable making changes to modules and building them again.
Aw man, thanks for posting this. Never thought building rom itself would take this much effort & resources. Rom devs are serioulsy awesome ppl. :good:
Thx a lot ,I just want to learn it,it is vevy clear and help me a lot
Thanks very much for creating this. I didn't try flashing the result yet, but the build finished without any problems.
thank you very much for your post, I also want to modify little bit in code aosp and test this changes. Could you plaese provide advice about how I can open code (Android studio?), do some changes and test it by emulator?
thanks for the great guide
but following it gapps will not be included in the build, correct?
do you guys know how to include open gapps?
tia!
hi everything worked in this guide in terms of the build. While flashing the device all steps succeed but during boot the pixel is stuck at the google loading screen. Any suggestions. I have been stuck on the screen for > 30 minutes.
rorlig said:
hi everything worked in this guide in terms of the build. While flashing the device all steps succeed but during boot the pixel is stuck at the google loading screen. Any suggestions. I have been stuck on the screen for > 30 minutes.
Click to expand...
Click to collapse
Maybe you did not download appropriate .sh scripts for your device or didn't run them successfully before building the code. These scripts additionally download files and without them you could have these problems which you mentioned.
I built and loaded AOSP Andorid 9 for PIxel 2 using the eng build vs the userdebug and its works however, when I start installing and granting Google services and such it works but i get a lot of crashes. do you have Google working and not crashing all the time?
```
$ adb root
$ adb remount
$ adb shell
$ cd /system/priv-app
$ mkdir GoogleServicesFramework
$ mkdir Phonesky
$ mkdir PrebuiltGmsCorePi
$ cp /sdcard/GoogleServicesFramework.apk GoogleServicesFramework/GoogleServicesFramework.apk
$ cp /sdcard/Phonesky.apk Phonesky/Phonesky.apk
$ cp /sdcard/PrebuiltGmsCorePi.apk PrebuiltGmsCorePi/PrebuiltGmsCorePi.apk
$ chmod 755 GoogleServicesFramework
$ chmod 755 Phonesky
$ chmod 755 PrebuiltGmsCorePi
$ chmod 644 GoogleServicesFramework/GoogleServicesFramework.apk
$ chmod 644 Phonesky/Phonesky.apk
$ chmod 644 PrebuiltGmsCorePi/PrebuiltGmsCorePi.apk
:: Need to add permissions for the three apps above
::If a device fails to boot, you need to logcat and grep for " - not in privapp-permissions whitelist" and add any missing items in the xml
$ adb push C:\Users\username\Desktop\PIxel2_9.0.0_eng_build\privapp-permissions-platform.xml /etc/permissions/privapp-permissions-platform.xml
```
Hello, I'm interested on the Mac os part. I've been building pixel experience on Ubuntu form am external HDD but because it's a 2011 iMac I have USB 2.0 and r/w speeds are really low slowering the whole process. On the internal drive I have a 500gb SSD that I'd like to use for compiling but partitioning is not an option, could you help me setting up enviroment?
PD: I tried setting it up with brew but I am missing dependencies I can't (don't know how) install them with brew, all guides are for Ubuntu or for Mac is but old.
Thank you in advance!
This guide inspired me to setup a Dockerized build and flash environment for the Pixel 5.
Leaving it here as Pixel 3 owners might find it useful: https://github.com/nvllsvm/pixel5-aosp-builder
Draje0 said:
This guide inspired me to setup a Dockerized build and flash environment for the Pixel 5.
Leaving it here as Pixel 3 owners might find it useful: https://github.com/nvllsvm/pixel5-aosp-builder
Click to expand...
Click to collapse
Thanks, this is very helpful! Have you tested the built image on a pixel 5?
ammarr said:
Thanks, this is very helpful! Have you tested the built image on a pixel 5?
Click to expand...
Click to collapse
Yup - it boots and seems to work except for phone call audio (T-Mobile US).
I am having issue, I did this and got:
#### build completed successfully (17:26:44 (hh:mm:ss)) ####
Click to expand...
Click to collapse
flashing claims to have succeeded but when the phone reboots it just goes back to fastboot mode and says "no valid slot too boot to"
The last few lines of output when doing "fastboot flashall -w" are:
Erase successful, but not automatically formatting.
File system type raw not supported.
Erasing 'metadata' OKAY [ 0.007s]
Erase successful, but not automatically formatting.
File system type raw not supported.
Rebooting OKAY [ 0.000s]
Finished. Total time: 82.933s
Click to expand...
Click to collapse
Is the filesystem raw not supported normal? Other than that I am really not sure why this isn't working.
very very handy post, appreciate it, even though i would probably be not building on my own. 20GB smh.. phew!
Anyone know or figure out how to get gapps on this once built (or built-in) without having to have TWRP?
Great guide. I am working on creating a custom rom myself. I've been wondering if it's possible to prevent system apps from being included in the build. There are a few apps that I use f-droid apps in their place (example K9 mail for stock email app) and don't want to see them re-appear when the ROM is updated. If this is not possible, can they be removed from the build before flashing?
Edit ..
Figured it out.
Hi.. I'm Building AOSP 10 for POCO F1(beryllium). i dont know which command should i choose in lunch cause my device isnt listed.. They have only for Pixel Devices.. Pls guide through it
***Key step: Download radio drivers.***
Most tutorials miss this or mention it very subtly. But, without this step the ROM you flash won't boot to the home screen (you will be in the boot loop).
Go to the driver binaries page, and download the right zip files for the Android build version (android-10.0.0_r10) and device (Pixel 3) you chose earlier in the repo command.
You will be downloading two zip files (one vendor image zip and one radio drivers zip), both zips will have on shell script file each (.sh), just put those two files in your repo folder (~aosp/pixel3) and run the scripts. It will download the required proprietary files (after asking you to accept the terms). Do not miss this step.. I lost 3 days trying to find the reason for my ROM not booting up, this was it.***
How can I get this 2 zip files for my Samsung device (SM-A715F). Thank you