Ok, so basically Im trying to extract data from my contacts2.db file.
Ive managed to copy it out of the hidden data folder on my GalaxySII onto my PC HDD. Now i need to run a script to extract the data. The thing is, im baffled how to do this.
Im a n00b when it comes to ADB, but i managed to drop the script and the copied contact2.db file into a test folder on my device, i managed to use the terminal emulator to browse to the directory and run the command but I get: 'Permission Denied' when I run the script. Ive tried changing the permissions on the file via my SII but to no avail.
So, im following the instructions I have via my PC:
== Usage ==
Copy your Android contacts database to your computer:
adb pull /data/data/com.android.providers.contacts/databases/contacts2.db ./
Run this file from the same directory as contacts2.db
./fb-extract.sh
Import csv to an address book of your choice
Now ive got the db, but how do I run the script. Ive tried to load adb emulator but have no idea what im doing there. Ive tried to use the adb command but again, no idea.
Can someone give me a brief, and quick step-by-step way to run this script on my contacts file so I can extract the data I want please.
Thanks muchly....
-=stylus=-
What are you trying to accomplish? The instructions say you should be running the shell script on the PC, not the phone. But you can't just natively run shell scripts in Windows, which I presume is what you're using. You'd need Cygwin or similar assuming you're averse to Linux, and possibly other stuff, depending on what the script is doing.
That fb-script you're trying to run is probably extracting the data with sqlite3. So you need to have the sqlite3 binary available where you are running the script (phone or computer). There are many things which can go wrong, first step would be to paste the content of fb-extract.sh here.
Ah, yes sorry. Im trying to extract all email addresses from my contacts on my phone. Facebook syncs with your Android device and adds all the email addresses registered with your contacts.
The script I have should extract all this data out:
(Not sure if Im allow to post code but)
#! /bin/sh
###########################################
#
# Android DB contacts exporter (proof-of-concept)
# 2011-01-06
#
# Extracts "certain contacts" from the Android contacts database to a CSV file
# for import into other contact list tools.
#
#
# == Usage ==
#
# 1) Copy your Android contacts database to your computer
# adb pull /data/data/com.android.providers.contacts/databases/contacts2.db ./
# 2) Run this file from the same directory as contacts2.db
# ./fb-extract.sh
# 3) Import csv to an address book of your choice
#
# == Notice ==
#
# This is a proof of concept script. Use at your own risk and check
# with laws and terms of service regarding data usage.
#
###########################################
# In and out files
CONTACTSDB="./contacts2.db"
OUTFILE="./contacts_fb.csv"
# Write table header
echo "First Name,Last Name,E-mail Address,Mobile Phone,Home Phone" >> $OUTFILE
# Get the raw IDs of all facebook contacts
RAWID_LIST=`sqlite3 $CONTACTSDB "SELECT _id from raw_contacts WHERE account_type='com.facebook.auth.login'"`
# Loop through the IDs and extract necessary data
for RAWID in $RAWID_LIST; do
# Get the first and last names
NAME=`sqlite3 $CONTACTSDB "SELECT data2, data3 FROM data WHERE raw_contact_id=$RAWID AND mimetype_id=7"`
FNAME=`echo $NAME | awk '{split($0,a,"|"); print a[1]}'`
LNAME=`echo $NAME | awk '{split($0,a,"|"); print a[2]}'`
# Get the email address
EMAIL=`sqlite3 $CONTACTSDB "SELECT data1 FROM data WHERE raw_contact_id=$RAWID AND mimetype_id=1"`
# Get the phone numbers (mobile & other). Convert international Japanese # to local using `sed`
MPHONE=`sqlite3 $CONTACTSDB "SELECT data1 FROM data WHERE raw_contact_id=$RAWID AND mimetype_id=5 AND data2=2" | sed 's/^81\([0-9]\{7,\}\)/0\1/'`
OPHONE=`sqlite3 $CONTACTSDB "SELECT data1 FROM data WHERE raw_contact_id=$RAWID AND mimetype_id=5 AND data2=7" | sed 's/^81\([0-9]\{7,\}\)/0\1/'`
# If at least one info field was set then write the csv line
if [ -n "$EMAIL" ] || [ -n "$MPHONE" ] || [ -n "$OPHONE" ]; then
echo "$FNAME,$LNAME,$EMAIL,$MPHONE,$OPHONE" >> $OUTFILE
fi
done
Now like I said, I have the contacts2.db file, I just am not sure how to go about running this script on it to accomplish my task.
I understand a bit of coding but not much so bear with me.
-=stylus=-
Well, it's obvious you have to run it on a Linux/Android system with /bin/sh and sqlite3 binaries. So, probably better to run it on your phone. But first, you need to have the sqlite3 binary from somewhere, as it's not shipped by default.
Get SuperOneClick and extract it; in the Dependencies folder you'll find the sqlite3 file.
Push it to the phone from your computer (these commands are run in command prompt):
Code:
C:\> adb remount /system
C:\> adb push /path/to/sqlite3 /system/xbin/sqlite3
Check the sqlite3 binary is present in the system path and available:
Code:
C:\> adb shell
[email protected]# which sqlite3
/system/xbin/sqlite3
Now you can run your script. With adb shell, run the script on your phone in the same folder where you have the .db file. Should create a new CSV file which can be pulled to your computer.
Now i just get the error message:
./fb-extract.sh: cannot execute - permission denied
chmod 777 fb-extract.sh
Depending on what folder you're in, you may need to remount r/w too.
And make sure you're not having the two files on a partition mounted with noexec
I wish I was good at this stuff....
Ok, its in a folder:
/sdcard/test
contents of this folder are:
contacts2.db
fb-extract.sh
Ive run the 'chmod 777 fb-extract.sh' command but still permission denied.
Should I put it in a different folder and run it? Is there a better way?
Sorry for my lack of knowledge here but no idea what remount r/w (well other than remount read/write) is or how to do it. How do I know if ive got it on a partition mounted with noexec?
In "adb shell":
Code:
[email protected] # mount | grep sdcard
and post the output. If you see a "noexec" flag, then it doesn't allow you to exec anything from the sdcard. You could try after that
Code:
[email protected] # mount -o remount,exec /sdcard
and try the script again.
I think im losing the will to live guys ha ha ha!!!
Ok the output was:
Code:
a/local/bin:$PATH <
[email protected]:/ $ su
[email protected]:/ # mount | grep sdcard
/dev/block/vold/259:3 on /mnt/sdcard type vfat (rw,dirsync,nosuid,nodev,noexec,noatime,nodiratime,uid=1000,gid=1015,fmask=0002,dmask=0002,allow_utime=0020,codepage=cp437,iocharset=iso8859-1,shortname=mixed,utf8,errors=remount-ro)
tmpfs on /mnt/sdcard/external_sd type tmpfs (rw,dirsync,nosuid,nodev,noexec,noatime,nodiratime,size=0k,mode=755,gid=1000)
tmpfs on /mnt/sdcard/usbStorage type tmpfs (rw,dirsync,nosuid,nodev,noexec,noatime,nodiratime,size=0k,mode=755,gid=1000)
/dev/block/vold/179:25 on /mnt/sdcard/external_sd type vfat (rw,dirsync,nosuid,nodev,noexec,noatime,nodiratime,uid=1000,gid=1023,fmask=0002,dmask=0002,allow_utime=0020,codepage=cp437,iocharset=iso8859-1,shortname=mixed,utf8,errors=remount-ro)
tmpfs on /mnt/sdcard/external_sd/.android_secure type tmpfs (ro,relatime,size=0k,mode=000) [email protected]:/ # mount -o remount,exec /sdcard
mount: can't find /sdcard in /proc/mounts
1|[email protected]:/#
Now im doing all this using terminal emulator on my phone. Thats the same as adb right? Still getting permission denied though.
You would think extracting email addresses from your contacts list was easy!
/mnt/sdcard type vfat (rw,dirsync,nosuid,nodev,noexec
This is what I was saying before.
On your phone with adb:
Code:
[email protected] # mount -o rw,remount /mnt/sdcard
and then you should be able to execute the script.
Well, it's difficult because it wasn't designed to be used that way (although what you are trying to do is quite simple). It would have been easier to sync your contacts with Google and download the CSV from Google Contacts' web interface.
Nope. Still cant do it.. still permission denied.
I cant sync with Google, create a vcard or anything. The facebook emails are protected somehow. If you export your contacts list to any file, it leaves all the facebook emails out!
Agghhhhhhh!
Thanks for all your help though. Much appreciated!!!
Related
DISCLAIMER: You know the risks, bricking, breaking etc. I'm not, nor is anyone else responsible if something goes wrong with your device, especially should it turn into a cave troll and bludgeon someone to death.
You will have to be rooted for this to work.
Update 1/5/11
Changed the install and loader scripts re-compressed and uploaded, new download link. Stream lined the installation process (removed 5 steps)
Update 1/4/11
reflect changes to ubuntu.sh and bootubuntu scripts and streamlining the process. Added screenshots
_________________________
I was over at nexusonehacks.net and looking at the ubuntu on nexus one hack and was thinking that this could be ported to the nook color. The idea is to run ubuntu from an img on the sd card.
The following is a modified version of the "how to" orginally posted here
http://nexusonehacks.net/nexus-one-hacks/how-to-install-ubuntu-on-your-nexus-oneandroid/
Credits
The guys at NexusOneHacks
xda user munday who has modified the scripts to work for the nook color
Devs at XDA Developer for the Ubuntu ARM image for HTC HD2
Saurik (Jay Freeman) for the Debian G1 script, which was hacked for Ubuntu on Nexus One! and then re-hacked for the Nook Color by munday
Here we go:
1) Go download the ubuntu file here: http://www.megaupload.com/?d=67BU4Y3T
2) Extract the contents using 7zip http://www.7-zip.org/download.html
3) Copy the extracted folder "ubuntu" to the root of your SD card
4) Unmount the nook from your pc, go into the app NookColor Tools (if you don't have it download it here courtesy of Ben 74 and click All Settings->Development then uncheck Auto Mount. This will prevent the SD card from being mounted automatically when plugged into the PC
5) Plug the Nook back into your pc and open a command line then navigate to your android sdk folder ie C:\android-sdk or wherever you have it and type
Code:
adb devices
Hit enter and make sure that the nook color comes up (should be the SN of your Nook)
Note: This can also be done from terminal emulator instead of adb...but it is kind of a pain
6) Now we want to go into the command line on the nook so type
Code:
adb shell
Hit enter and this should bring up a "#" sign.
7) Type
Code:
su
hit enter (I do this just as a force of habit)
8) Now we navigate to the ubuntu folder we just copied to our sd card by entering
Code:
cd /sdcard/ubuntu
hit enter. To confirm you are where you want to be you can type
Code:
ls
and it should list the contents of the ubuntu folder.
9) Run the setup script ubuntu.sh by entering the following:
Code:
sh ./ubuntu.sh
hit enter
10) Now type
Code:
bootubuntu
and hit enter
11) So now we have ubuntu command line up and running which should look like:
Code:
[email protected]:/#
We will need to get some packages to run a gui and then connect to it via VNC client. So we need to update a few things first. Type:
Code:
apt-get update
then
Code:
apt-get install tightvncserver
now we need the gui elements so enter this:
Code:
apt-get install lxde
12) Type the following:
Code:
export USER=root
vncserver -geometry 1024×600
this sets the resolution...you can change it to play around, but the native resolution for the nook is 1024x600. This is also where you will set the password for your vnc server...make sure to remember it
13) Next, we are going to add the following to /root/.vnc/xstartup file using cat command: (hit enter after each line and ignore the fact that it looks like it does it twice)
Code:
cat > /root/.vnc/xstartup
#!/bin/sh
xrdb $HOME/.Xresources
xsetroot -solid grey
icewm &
lxsession
After hitting enter after the last line press "Ctrl+D" twice then press enter
14) We need the vnc server to start every time we start ubuntu, type this:
Code:
cat > front
export USER=root
cd /
rm -r -f tmp
mkdir tmp
cd /
vncserver -geometry 1024×600
Same as before, hit enter after each line then after hitting enter for the last line press "Ctrl+D" twice then press enter.
15) Enter this
Code:
cat front /root/.bashrc > temp
Then
Code:
cp temp /root/.bashrc
16) Now on the nook open up your vnc viewer app (android vnc viewer or PocketCloud...or whatever you use)
Use the following settings:
Host Address: 127.0.0.1 (some don't require this)
Password: the password you set up in step 16
Port: 5901
Save and then click connect and you should be up and running on ubuntu! To leave ubuntu type exit from adb. This should kill ubuntu. Exit adb and then you can disconnect the nook from your pc.
To start ubuntu from the nook open up a terminal emulater and type:
Code:
bootubuntu
Hit enter, this should bring up the [email protected], hhen press "Home" key and open up your vnc app. Now you are ready to rock your Linux tablet!
You could also use gscript lite to automate the process of starting ubuntu so you don't have to type it in the terminal everytime. You can get gscript lite from the market or download the file I have attached.
It does seem a little laggy..just a little...for now anyway
To exit ubuntu: Disconnect the vnc client then hold the "Home" key (I use button savior which brings up a nice little task manager) and click the Terminal emulator when it pops up under running tasks. Once you are back in the terminal you can type:
Code:
exit
to shut down ubuntu.
Enjoy and make sure you thank munday and the guys at nexusonehacks and all the devs who helped make it possible.
-devastator
EDIT:
I noticed that after rebooting the nook would hang at the "Future of reading" screen....if this happens turn the device off remove the sd card the turn the nook back on. It should boot fine. After it's finished you can return the sd card and mount it. This is strange and I'm looking into it. I'm not sure if this is limited to me or if it is universal to all nooks. It also still fails at shutting ubuntu down for some reason. I'm still looking into this.g into this.
I got it going, just needed a few "busybox"-es here and there . It needs some work, right now to get X you have to use VNC.
It's sluggish but a bit more work might just yield something a bit more usable. I'll play more tomorrow night.
Here's my scripts:
ubuntu.sh
Code:
mount -o remount,rw /dev/block/mmcblk0p5 /system
busybox modprobe ext2
rm -f /data/local/bin/fsrw
rm -f /data/local/bin/bootubuntu
rm -f /data/local/bin/unionfs
rm -f /data/local/bin/installer.sh
rm -f /data/local/bin/mountonly
mkdir /data/local/mnt
#busybox clear
rm /system/bin/fsrw
rm /system/bin/bootubuntu
rm /system/bin/unionfs
rm -/system/bin/mountonly
busybox cp -f fsrw /system/bin
busybox cp -f bootubuntu /system/bin
busybox cp -f unionfs /system/bin
busybox cp -f mountonly /system/bin
cd /sdcard/ubuntu
busybox chmod 4777 *
cd /system/bin/
busybox chmod 4777 *
cd /
#busybox clear
echo " "
echo "Ubuntu Chroot Bootloader v0.1"
echo "Ubuntu Bootloader is now installed!"
echo "This process does NOT damage Android OS!"
echo " "
echo "Original Installer by Charan Singh"
echo "Modified for Ubuntu Chroot by Max Lee at NexusOneHacks.net"
echo " "
echo "To enter the Debian Linux console just type 'bootubuntu'"
bootubuntu
Code:
mount -o remount,rw /dev/block/mmcblk0p5 /system
export kit=/sdcard/ubuntu
export bin=/system/bin
export mnt=/data/local/mnt
export PATH=$bin:/usr/bin:/usr/sbin:/bin:$PATH
export TERM=linux
export HOME=/root
busybox mknod /dev/loop2 b 7 0
mount -o loop,noatime -t ext2 $kit/ubuntu.img $mnt
mount -t devpts devpts $mnt/dev/pts
mount -t proc proc $mnt/proc
mount -t sysfs sysfs $mnt/sys
busybox sysctl -w net.ipv4.ip_forward=1
echo "Setting /etc/resolv.conf to Google Open DNS 8.8.8.8 and 8.8.4.4"
echo "nameserver 8.8.8.8" > $mnt/etc/resolv.conf
echo "nameserver 8.8.4.4" >> $mnt/etc/resolv.conf
echo "Setting localhost on /etc/hosts "
echo "127.0.0.1 localhost" > $mnt/etc/hosts
echo "READY TO ROCK AND ROLL BABY! "
echo "Brought to you by NexusOneHacks.net and the open source community! "
echo " "
busybox chroot $mnt /bin/bash
#After exit command is executed clear it all up
echo " "
echo "Shutting down Ubuntu"
umount $mnt/dev/pts
umount $mnt/proc
umount $mnt/sys
umount $mnt
Then just followed the instructions at nexusonehacks.
note: to run bootubuntu i had to use sh and the use the full path:
Code:
# sh /system/bin/bootubuntu
other note: I was using adb not a term app on the nook.
Obligatory video
I can't post links yet so check: youtube.com/watch?v=TnXfZ6aagn8 for a really boring video
EDIT: switched the names, DOH!
I also had to use sh and the full path also. Thanks for the update on the script...gonna try it in the morning (it's 2am here) will also see if I can help to get it running smoother. What class SD card are you using? I imagine a class 4 or higher would yeild better results as far as responsiveness goes.
The sd is a class 2 that came in one of my dev phones. I'll pick up a faster one tomorrow after work and test it out.
It's just about 3am here now, we must be in the same time zone.
I'm going to bed now, but first here's a modded version of the script you had originally posted that also works now.
bootubuntu
Code:
#!/bin/sh
mount -o remount,rw /dev/block/mmcblk0p5 /system
export kit=/sdcard/ubuntu
export bin=/system/bin
busybox [ ! -d /data/local/ubuntu ] && mkdir /data/local/ubuntu
export mnt=/data/local/ubuntu
export PATH=$bin:/usr/bin:/usr/sbin:/bin:$PATH
export TERM=linux
export HOME=/root
busybox mknod /dev/loop1 b 7 0
busybox losetup /dev/block/loop1 /sdcard/ubuntu/ubuntu.img
mount -t ext2 /dev/block/loop1 /data/local/ubuntu
#mount -o loop,noatime -t ext2 $kit/ubuntu.img $mnt
mount -t devpts devpts $mnt/dev/pts
mount -t proc proc $mnt/proc
mount -t sysfs sysfs $mnt/sys
busybox sysctl -w net.ipv4.ip_forward=1
echo "Setting /etc/resolv.conf to Google Open DNS 8.8.8.8 and 8.8.4.4"
echo "nameserver 8.8.8.8" > $mnt/etc/resolv.conf
echo "nameserver 8.8.4.4" >> $mnt/etc/resolv.conf
echo "Setting localhost on /etc/hosts "
echo "127.0.0.1 localhost" > $mnt/etc/hosts
echo "READY TO ROCK AND ROLL BABY! "
echo "Brought to you by NexusOneHacks.net and the open source community! "
echo " "
busybox chroot $mnt /bin/bash
#After exit command is executed clear it all up
echo " "
echo "Shutting down Ubuntu"
umount $mnt/dev/pts
umount $mnt/proc
umount $mnt/sys
umount $mnt
busybox losetup -d /dev/loop1
#unregistering the loopback device just seems to die. need to fix or remove.
I'll look at the N1 formus tomorrow too and see if they have made progress in getting X to run fast or without need for VNC.
Nice work, I am addicted in seeing the crazy crap people are doing everyday.
Couldn't sleep so I'm messing around with it now...I'm using a class 4 and have it up and running. It does seem to be a little laggy, but I don't have a class 2 to compare it to. Let me know what you find out, I'll keep playing with it also to see if I can get it to be more responsive...but we are definately on our way to having a great little linux tablet in addition to android. You should also be able to use this for other builds of linux also by just changing out the .img file to a flavor of your choosing. I'm excited
I have also updated the OP with the how to and credits. Really appreciate your help munday.
I noticed that after rebooting the nook would hang at the "Future of reading" screen....if this happens turn the device off remove the sd card the turn the nook back on. It should boot fine. After it's finished you can return the sd card and mount it. This is strange and I'm looking into it. I'm not sure if this is limited to me or if it is universal to all nooks.
Edit:
Running into some strange occurrences...it appears that when you type exit from the terminal to kill ubuntu, I believe ubuntu stays running...anyway, I'm going to bed and will work on it more tomorrow.
devastatorx said:
You should also be able to use this for other builds of linux also by just changing out the .img file to a flavor of your choosing. I'm excited
Click to expand...
Click to collapse
Are there any increased risks with changing the version you use?
Well, you will want to make sure that the build supports ARM based devices.
munday said:
Then just followed the instructions at nexusonehacks.
note: to run bootubuntu i had to use sh and the use the full path:
Code:
# sh /system/bin/bootubuntu
Click to expand...
Click to collapse
I fixed this, the scripts were backwards, I have updated the OP to reflect the changes and bootubuntu now works
devastatorx said:
I fixed this, the scripts were backwards, I have updated the OP to reflect the changes and bootubuntu now works
Click to expand...
Click to collapse
sweet, i can't wait to get home to play more!!!
I've tried running ubuntu.sh from the terminal emulator (With su) and from ADB, and have not had success with either.
The readout is:
Code:
': File existste '/system/bin
': File existste '/system/bin
': File existste '/system/bin
: not found
: not found
cd: can't cd to /sdcard/ubuntu
: not found
: No such file or directory
: not found
cd: can't cd to /system/bin
: not found
: No such file or directory
: not found
cd: can't cd to /
: not found
: not found
Ubuntu Chroot Bootloader v0.1
Ubuntu Bootloader is now installed!
etc., etc.
Is there a problem with how I copied the script? I assume the first part about the file existing is due to this being the third or fourth time I've tried running the script (Unsuccessfully), but I've no idea how to interpret the rest.
Thanks
munday said:
sweet, i can't wait to get home to play more!!!
Click to expand...
Click to collapse
I sent you a message
Link3737 said:
I've tried running ubuntu.sh from the terminal emulator (With su) and from ADB, and have not had success with either.
The readout is:
Code:
': File existste '/system/bin
': File existste '/system/bin
': File existste '/system/bin
: not found
: not found
cd: can't cd to /sdcard/ubuntu
: not found
: No such file or directory
: not found
cd: can't cd to /system/bin
: not found
: No such file or directory
: not found
cd: can't cd to /
: not found
: not found
Ubuntu Chroot Bootloader v0.1
Ubuntu Bootloader is now installed!
etc., etc.
Is there a problem with how I copied the script? I assume the first part about the file existing is due to this being the third or fourth time I've tried running the script (Unsuccessfully), but I've no idea how to interpret the rest.
Thanks
Click to expand...
Click to collapse
Try using these, just unrar and then copy them in your ubuntu folder, if it asks to overwrite just click yes...also you have autmounting of the sd card turnned off in nook color tools settings right?
Yeah, it must have been an error on my end. Using the updated scripts you posted worked like a charm.
Thanks!
Link3737 said:
Yeah, it must have been an error on my end. Using the updated scripts you posted worked like a charm.
Thanks!
Click to expand...
Click to collapse
Good to hear!
I'm currently working on making this a port to the SD card so that with the sd card in it boots into ubuntu from power off, and with the sd card out it boots to android then you replace the sd and use it for storage like normal. It will also run a lot snappier....well that's the idea anyway.
devastatorx said:
Good to hear!
I'm currently working on making this a port to the SD card so that with the sd card in it boots into android from power off, and with the sd card out it boots to android then you replace the sd and use it for storage like normal. It will also run a lot snappier....well that's the idea anyway.
Click to expand...
Click to collapse
Booting from SD in the same fashion as Nookie Froyo? That sounds like it would give a huge performance boost over the current method, I can't wait to see what you come up with.
This should be acheivable using the method mentioned in this topic:
http://forum.xda-developers.com/showthread.php?t=873243 thanks to clockworx.
Currently looking into this
devastatorx said:
This should be acheivable using the method mentioned in this topic:
http://forum.xda-developers.com/showthread.php?t=873243 thanks to clockworx.
Currently looking into this
Click to expand...
Click to collapse
I was thinking the exact same thing, the nook seems to like booting from sd, so we should be able to set up the partitions like the foryo sd and edit the init.rc and env.txt to boot ubuntu instead of android. Just a theory though. It's my plan for tonight anyway.
munday said:
I was thinking the exact same thing, the nook seems to like booting from sd, so we should be able to set up the partitions like the foryo sd and edit the init.rc and env.txt to boot ubuntu instead of android. Just a theory though. It's my plan for tonight anyway.
Click to expand...
Click to collapse
I also stumbled across this: http://androix.org/ Which talks about a native android xserver which does a way with the vnc aspect altogether. I downloaded and attempted to install but I get the error "There is a problem parsing the package" and I haven't had any luck plus there isn't much documentation.
Let me know how it goes munday, I'll be working on the same thing later (have to take the fiance out for dinner)
question
I was just curious, after installing ubuntu can we install flash as we would had we booted linux from netbook?
All About Contacts Import/Export, Backup/Restore and Data/Sync
Last Change: 2011-12-29
Content:
Details about how to backup and restore Contacts on the
Samsung Galaxy S I & II, including data storage info.
Summary:
Backing up and restoring your contacts on the latest Android OS
based phones is a nightmare as long as you do not want to use
automatic sync via Google or Outlook Express. In fact without
paying money for some specialized Application to do it properly
for you, it is a mess. Here I feel Samsung have really failed.
Why there isn't already a built-in simple-to-use contact backup
functionality, that doesn't depend on Google, Kies, Bluetooth
or other net-based service, is way beyond me.
Anyway, here is my attempt to remedy this situation and the
various options you have to backup and restore all your contacts
without:
a) using Google, Microsoft, Kies or other wireless protocol and
b) paying for specialized applications or software.
To simplify this exercise (a lot) we will assume that we would be
satisfied with backup and restore of only the most essential information.
This means (for me):
- first_name
- last_name
- primary_phone
- primary_email
So let's see how far we can get...
NOTE:
a) All of this tutorial assumes you have a rooted phone and
that you are operating as the root user (su)!
b) AOS phones doesn't generally come with the SQLite3
command-line binary, so this have to be installed manually!
Keywords:
Contacts, Restore, CSV, vCard, SQL,
Backup, Data, Sync, SQLite3, Import, Export, VCF
Content - Introduction
- Remounting /system as Read/Write
- Installing the SQLite Command-Line Interpreter
- SQLite3 Install Script
- Contacts Database & Storage-Location:
- Contacts DB Location List
- Direct Manipulation of the Contacts Database
- Other Contact Import / Export Applications
- Help Needed!IntroductionEssentially there are only 4 different ways to backup your Android contacts.
1. By using a backup application like Kies or 3rd party one from Android Market.
2. By syncing your contacts to your Google account.
3. By copying the entire "Contacts" database to your PC.
4. By exporting only the relevant data directly on your phone, using SQLite.
In this post I will only talk about the last 2 options, with focus on (4) as it applies to (3) as well.
Remounting /system as Read/WriteAny manipulation of the system partitions and the root directory require WRITE permission to the file-system. To find out, just use the mount command and look for the "rw" label in the filesystem description of the / and /system partitions. For example:
Code:
# mount
...
rootfs / rootfs ro,relatime 0 0
/dev/block/mmcblk0p9 /system ext4 ro,relatime,barrier=1,data=ordered 0 0
/dev/block/mmcblk0p10 /data ext4 rw,nosuid,nodev,noatime,...,discard 0 0
/dev/block/mmcblk0p4 /mnt/.lfs j4fs rw,relatime 0 0
/dev/block/vold/179:11 /mnt/sdcard vfat rw,dirsync,...,utf8,...,discard 0 0
...
See the "rw" following "/system ext4" and "/data ext4" above?
Yes? Good, you can skip to the next section!
No? Then you need to remount the filesystem as READ/WRITE.
("rw" = "Read AND Write" and "ro" = "Read Only".)
!! However, this may depend on your model AND on how you connected your phone to USB. You can enter the phone directly without using the "file-transfer" mode in which case you may or may not have RW, but if you select to use "file-transfer" (aka. USB-disk mode) you may only get RO. In some cases the /mnt/sdcard is simply never mounted. Try and test! [*** check this! ***]
The way to get RW, is to remap the "physical" device (disk) location to the system partition you are interested in. The tricky part is knowing the physical device name of your system disk which may not always be the same as indicated from the mount command. If not, try to inspect the boot-log messages with the "motd" command, or inspecting the file "/init.rc" for the details of everything that is mounted upon boot. (Boot-log messages are normally enabled in most stock Kernels, but not necessarily in modded ROM's.)
For the SGS-2 which uses the EXT4 filesystem, I had to use:
Code:
mount -o remount,rw -t ext4 /dev/block/mmcblk0p9 /system
For the SGS-1 when using an RFS based AOS, I had to use:
Code:
mount -o remount,rw -t yaffs2 /dev/block/mtdblock3 /system
To undo this you just replace "rw" with "ro" in the statements above. Although this is not necessary, since it will be automatically reverted after next reboot, which is recomended anyway after manually editing any system DB.
Installing the SQLite Command-Line InterpreterMost phones doesn't come with the SQLite3 command-line interpreter/interface (CLI) installed, unless they are rooted with a heavily modded ROM. The CLI is needed to manually access the SQLite Contacts database. Thus you need to check that it is not already installed and install it manually, if it is not. The binary (sqlite3) is normally located in either /system/bin or /system/xbin. To check if you have it, try the following command:
Code:
find / -iname '*sqlite*'
If it fails to return the files shown below, you need to install the SQLite3 binary AND possibly the relevant library files. You should already have most of the required library (lib*.so) files, as they are constantly used by the system, so look for them first! The files you need are:
/system/bin/sqlite3
/system/lib/libsqlite.so
/system/lib/libsqlite_jni.so
If the binary is compiled from CyanogenMod sources, you will also need the following:
/system/lib/libncurses.so
In the worst case you can go through the exercise to compile the SQLite3 sources for the MIPS32 chipset by yourself. [http://www.sqlite.org/] Also it's worth to note that some custom ROM's now include the latest compiled version. When you have the binary you just put it on your SD card and copy it to the root file system binary directory. Many ROM's and rooted phones use an additional directory /system/xbin where you can put your own binaries. If this directory (and path variable) is not already present, use:
Code:
cp <path_to_your_sqlite3_binary> /system/bin/sqlite3
chmod 4755 /system/bin/sqlite3
chown root.shell /system/bin/sqlite3
(Note that this version of chown uses a "." in the "user.group" specification.)
Do the same for the library files, if they are missing. You should now be able to run SQLite3:
Code:
# sqlite3 -version
3.6.22
Everything OK? Good!
SQLite3 Install Script!!MODIFY FIRST!!
Code:
#!/bin/sh
# squint.sh - sqlite3 install script for rooted Androids
#
#-rwsr-sr-x root shell 22228 2011-11-10 12:53 /system/bin/su
#-rwsr-xr-x root shell 1075144 2011-12-15 16:55 /system/xbin/busybox
#-rw-r--r-- root root 322120 2011-09-14 14:45 /system/lib/libsqlite.so
# Go to your "sqlite" folder on your (internal) SD card:
cd /mnt/sdcard/sqlite/
cp sqlite3 /system/xbin/sqlite3
chmod 4755 /system/xbin/sqlite3
chown root.shell /system/xbin/sqlite3
cp ./lib/libncurses.so /system/lib/libncurses.so
chmod 644 /system/lib/libncurses.so
chown root.root /system/lib/libncurses.so
Contacts Database & Storage-LocationApparently the SQLite3 database used to store the Contacts data is located in different parts on different Android models [?] and different Android OS versions. I have not been able to verify this, as I don't have access to these phones, but from from reading various forum posts for other phones this seem to be the case.
The way you can check is by looking for the raw database file on your system. On the phones I have, the location can be found here:
SGS-1 (2.3.6): /dbdata/databases/com.android.providers.contacts/contacts.db
SGS-2 (2.3.4): /data/data/com.android.providers.contacts/databases/contacts2.db
It may be important to notice that the "com.android.providers.contacts" part of this path, is actually referring to the particular "Contacts.apk" application that is installed in your phone. Therefore this location can differ from other versions of that application and thus also your phone model and OS version. In addition note that the /data and /dbdata or "whatever" partitions may be mounted on a different block device, as noted in the first section above. For example, on my SGS-2, I had /data mounted on /dev/block/mmcblk0p10 .
The more intelligent way is to look for all .db files on you system by doing some shell magic:
Code:
find / -iname '*.db'
But today, almost every application is using a database to save information, so to narrow down the long list, filter your search with "ontact":
Code:
find /system -iname '*.db' |grep ontact
Contacts DB Location ListGT-I9000 (2.3.6): /dbdata/databases/com.android.providers.contacts/contacts.db
GT-I9100 (2.3.4): /data/data/com.android.providers.contacts/databases/contacts2.db
[I'd like to keep this list updated with other models and AOS versions, so please let me know what you have.]
Direct Manipulation of the Contacts DatabaseIn order to make any changes to any information contained in the database, you need to have write permissions set on the the relevant partition(s) as explained above. Remount as needed.
If you for some crazy reason need to replace/restore your entire contact database with another one. (This is strongly discouraged!) You also need to make sure that the one you replace with have both the same ownership (chown) and permissions (chmod) as the original one.
Next, let's open the contacts database:
Code:
# cd /data/data/com.android.providers.contacts/databases
# sqlite3 contacts2.db
SQLite version 3.6.22
Enter ".help" for instructions
Enter SQL statements terminated with a ";"
sqlite> .databases
.databases
seq name file
--- --------------- ----------------------------------------------------------
0 main /data/data/com.android.providers.contacts/databases/contacts2.db
sqlite> .tables
.tables
_sync_state settings
_sync_state_metadata speed_dial
accounts status_updates
activities v1_settings
agg_exceptions view_contacts
android_metadata view_contacts_restricted
calls view_data
contact_entities_view view_data_restricted
contact_entities_view_restricted view_groups
contacts view_raw_contacts
data view_raw_contacts_restricted
groups view_v1_contact_methods
mimetypes view_v1_extensions
name_lookup view_v1_group_membership
nickname_lookup view_v1_groups
packages view_v1_organizations
phone_lookup view_v1_people
properties view_v1_phones
raw_contacts view_v1_photos
sqlite> SELECT _id,name, number FROM view_v1_phones;
[...]
To output the result as a CSV table to a file called "my_contacts_db.csv" with the first row containing the (column) headers and bailing out on errors, do this:
Code:
sqlite> .bail ON
sqlite> .headers ON
sqlite> .mode csv
sqlite> .output my_contacts_db.csv
sqlite> SELECT _id,name, number FROM view_v1_phones;
sqlite> .output stdout
sqlite> .quit
As you can see, if we had known the precise query statements we would have already been done with our export. But the Contacts.db tables are rather complex for our purposes of simply extracting the most relevant contact information. I lazily gave up after trying some basic SQL select statements...
[So yeah, please send me a good SQL statement for selecting and printing only the basics as a CSV!]
The AOS java API for using the contacts are MUCH simpler to use than the needed SQLite statements, for doing the extractions/replacements. But then you also need to write the program.
http://www.higherpass.com/Android/Tutorials/Working-With-Android-Contacts/
There are many such programs out there, but very few who does the job well. (Everybody want to make money!) So try some Google-Fu on sites such as:
http://www.sourceforge.com/
http://code.google.com/
http://www.github.com/
http://www.stackexchange.com/
http://grepcode.com/
and use the keywords in the top of this document.
Other Contact Import / Export ApplicationsBut I'm not a Java Programmer. (What?!!) So instead I had a look at the most simple applications for exporting and importing you contacts. Being reasonably paranoid as a person, I did not trust huge multifunction backup-applications, knowing that they are full of adware, spyware and other "things" we may not like. With these limits I knew that any such application should not be much larger than 100 kilobytes.
I found 4 small applications (and one very large) on the Market that I tested:1. Contacts CSV Export (93 K)
https://market.android.com/details?id=dutchandroid.contacts.export
(contacts.export-1060-1.060.apk, dutchandroid.contacts.export)
2. Contacts Backup Trial (168K)
https://market.android.com/details?id=com.saturnat.android.contactsbackup.trial
(contactsbackup.trial-34-2.9.5.apk, com.saturnat.android.contactsbackup.trial)
3. Import Contacts (56 K)
https://market.android.com/details?id=org.waxworlds.edam.importcontacts
(import-contacts_1.1.apk, org.waxworlds.edam.importcontacts)
4. Offline Contacts Sync (27 K)
https://market.android.com/details?id=com.beforemadness
(offlineContactsSync.apk, com.beforemadness)
5. GO Contacts EX (2.7 M!)
https://market.android.com/details?id=com.jbapps.contactpro(1) Is a flexible lightweight CSV contacts exporter. You can manually choose CSV delimiters (",;TAB" etc) or choose a standard (Mozilla, Hotmail and Outlook etc.) CSV format.
(2) $$$, Is buggy with a no-sense UI, doesn't catch Email addresses, don't let you choose filename, and contain trackers from Facebook and Mixpanel. Limited trial to 10 contacts and 3 days before expiration.
(3) Only imports vCard (.vcf) files. One-by-one or scan a directory. Ask's what to do (skip,merge,replace) with duplicates.
(4) No-sense UI. Has export function but it is not working. And import is only working partially.
(5) Exports all contacts to a single vCard file, but missed my Emails. Nice merge and batch-delete functionality. Nice interface, and works. However, this app is huge, and god knows what other bloat junk it contains.
Other Useful Links[1] http://www.sqlite.org/
[2] http://www.androidadb.com/source/pl...ndroid/contacts/ImportVCardActivity.java.html
[3] http://grepcode.com/project/repository.grepcode.com/java/ext/com.google.android/android/
[4] sed Tutorial: http://www.grymoire.com/Unix/Sed.html
Help Needed!If you have any detailed knowledge about where and how the relevant contact data is stored on your device, please let me know and include details such as:
- Your phone model number
- Your general Android OS version [Settings --> About Phone ]
- Your phone firmware version (PDA / PHONE / CSC) ["dial" --> *#1234#]
- The precise location of the contacts database file
E.g.
GT-I9100, Gingerbread 2.3.4, XWKI4/XXKI1/OXX KI2 (SEB)
/dbdata/databases/com.android.providers.contacts/contacts.db
If you have already been poking around your OS it would also be helpful to know:
- The version and type of database used (SQLite3 etc.)
- The particular DB tables used to obtain:
+ First Name
+ Last Name
+ Phone Number "1"
+ Email "1"
Here I have written "1" to signify the primary number used. I have refrained from using more details as this, as I consider this minimal but sufficient for backing up and restoring my contacts.
Some useful SQLite queries
Code:
sqlite3 -csv contacts2.db "SELECT _id,name,primary_email,number FROM view_v1_phones;"
sqlite3 -csv contacts2.db "SELECT _id,mimetype_id,raw_contact_id,data1 FROM data;"
sqlite3 -csv contacts2.db "SELECT _id,display_name FROM raw_contacts WHERE account_type='vnd.sec.contact.phone';"
sqlite3 -csv contacts2.db "SELECT data1 FROM data WHERE raw_contact_id=112 AND mimetype_id=5;"
sqlite3 -csv -header contacts2.db "SELECT _id,package_id,mimetype_id,raw_contact_id,is_primary,is_super_primary,data_version,data1,data2,data3 FROM data WHERE raw_contact_id=21;"
sqlite3 -csv -header contacts2.db "SELECT _id,package_id,mimetype_id,raw_contact_id,is_primary,is_super_primary,data_version,data1,data2,data3 FROM data WHERE is_primary=0 AND data_version=0;"
#SELECT data1 FROM data WHERE raw_contact_id=140 AND mimetype_id=1;
#SELECT data1 FROM data WHERE raw_contact_id=140 AND mimetype_id=5 AND data2=2;
#SELECT data1 FROM data WHERE raw_contact_id=112 AND mimetype_id=5 AND data2=7;
#SELECT data1 FROM data WHERE raw_contact_id=112 AND mimetype_id=5 AND data2=1;
If this helped you in any way, please hit the THANK YOU button!
The Android contacts-database extraction script
This script is based on one that I found for extracting Facebook contacts into a CSV file, here: http://forum.xda-developers.com/showthread.php?t=1170765 I then modified and improved this script to suit my own purposes, but I am very grateful to whoever created it in the first place. (There is no reliable reference to the origin of this.)
However, there are two very useful things I found and included in this script.
1) You can extract contacts from other applications using that same database file, by modifying the "java-path-identifyer" accordingly.
Code:
Specify the application used for organizing the data:
-----------------------------------------------------------
vnd.sec.contact.phone - Default Android phone book
com.facebook.auth.login - Facebook contacts
2) The sed pipe used to replace carrige-returns (new-lines, \n) from any output stream, with a comma ",".
Code:
sed ':a;N;$!ba;s/\n/,/g'
Explanation:
Create and label a register via :a
Append the current and next line to the register via N
If we are before the last line, branch (b) to the created
register (a) with $!ba ( "$!" means not to do it on the
last line, as there should be one final newline).
Use global match (/g) to substitute (s) every newline (\n)
with a comma on the pattern space, which is the contents
of the (a) register. (I.e. The whole input stream.)
Here is the script:
Code:
#!/bin/sh
# contex.sh - Extracts the Name,Phone,Email from the Android Contact Database
#==============================================================================
# Last Change: 2011-12-29
# Name: Android DB Contacts Exporter
#
# Description: Extracts contacts from the Android contacts database
# to a CSV file for import into other contact list tools.
#
# Usage: ./contex.sh <output_filename> &
#
# How to use:
#
# 1. Copy your Android Contacts Database to your computer:
# adb pull /data/data/com.android.providers.contacts/databases/contacts2.db ./
# 2. Run this file from the same directory as contacts2.db
# ./contex.sh output_filename.csv
#
# System Requirements: sh, sqlite3, sed
#
# Additional Notes:
#
# 1. This script is very slow, please be patient!
#
# 2. The sed sequence to replace the end on line (\n) with a comma (,)
# from a list, but avoiding \n on the last line is:
#
# sed ':a;N;$!ba;s/\n/,/g'
# --- OR ---
# sed '{:q;N;s/\n//g;t q}'
#
# 3. Only numbers that are of the types: HOME, MOBILE, WORK and OTHER
# get extracted in this script. These types are specified by the
# "data2" field in the "data" table.
#
#==============================================================================
echo; echo "Android contacts backup script started with PID: $$ "; echo;
DATE=`date +%G%m%d`
#----------------------------------------------------------
# Input and Output files:
#----------------------------------------------------------
CONTACTSDB="./contacts2.db"
if [ ! -f "$CONTACTSDB" ]
then
echo "Contacts Database file: $CONTACTSDB not found. Exiting."; echo
exit 1
fi
OUTFILE=$1
if [ -z "$OUTFILE" ]
then
OUTFILE="contex_backup_${DATE}.csv"
fi
echo "Contacts backup file: $OUTFILE "; echo
if [ -f "$OUTFILE" ]
then
echo "Contacts Backup file: $OUTFILE already exist. Exiting."; echo
exit 1
fi
#----------------------------------------------------------
# Write the Table Header:
#----------------------------------------------------------
echo "First Name,Last Name,E-mail,Mobile Phone,Home Phone,Work Phone,Other Phone" >> $OUTFILE
#----------------------------------------------------------
# Setting up SQLite command and mode:
#----------------------------------------------------------
#CMD="sqlite3 -bail -noheader -csv"
CMD="sqlite3";
#----------------------------------------------------------
# Get the raw IDs of all contacts:
#----------------------------------------------------------
RAWID_LIST=`$CMD $CONTACTSDB "SELECT _id from raw_contacts WHERE account_type='vnd.sec.contact.phone'"`
#echo "$RAWID_LIST"
#----------------------------------------------------------
# Loop through the IDs and extract all necessary data
#----------------------------------------------------------
for RAWID in $RAWID_LIST; do
# Get the First_Name and Last_Name in one query:
# NOTE: Need to remove occasional CTRL-M's with sed.
NAME=`$CMD -csv $CONTACTSDB "SELECT data2, data3 FROM data WHERE raw_contact_id=$RAWID AND mimetype_id=6;" | sed ':a;N;$!ba;s/\n//g'`
# Get the Email address:
# NOTE: This will simply append multiple email adresses to one long string, with ";" separator!
EMAIL=`$CMD $CONTACTSDB "SELECT data1 FROM data WHERE raw_contact_id=$RAWID AND mimetype_id=1;" | sed ':a;N;$!ba;s/\n/;/g'`
# Here we get the "Home", "Mobile", Work" and "Other" (labeled) phone numbers.
# NOTE: Each of these may return multiple responses, so we need to convert new-lines (\n) to ",".
HPHONE=`$CMD $CONTACTSDB "SELECT data1 FROM data WHERE raw_contact_id=$RAWID AND mimetype_id=5 AND data2=1;" | sed ':a;N;$!ba;s/\n/,/g'`;
MPHONE=`$CMD $CONTACTSDB "SELECT data1 FROM data WHERE raw_contact_id=$RAWID AND mimetype_id=5 AND data2=2;" | sed ':a;N;$!ba;s/\n/,/g'`;
WPHONE=`$CMD $CONTACTSDB "SELECT data1 FROM data WHERE raw_contact_id=$RAWID AND mimetype_id=5 AND data2=3;" | sed ':a;N;$!ba;s/\n/,/g'`;
OPHONE=`$CMD $CONTACTSDB "SELECT data1 FROM data WHERE raw_contact_id=$RAWID AND mimetype_id=5 AND data2=7;" | sed ':a;N;$!ba;s/\n/,/g'`;
PHONES="$MPHONE,$HPHONE,$WPHONE,$OPHONE";
# If at least one main field is found then write the CSV line
if [ -n "$NAME" ] || [ -n "$EMAIL" ] || [ -n "$PHONES" ]; then
#echo "ID: $RAWID"
echo -n "$NAME,$EMAIL,$PHONES" >> $OUTFILE
echo >> $OUTFILE
fi
done
echo 'Done.'
NOW YOU CAN HIT THE THANK YOU BUTTON!
reserved for me 2!
Thanks for this write up. I found my stuck after upgrading a friends phone from 2.2 firmware to CM10 with 4.1 firmware. We made a full backup with TitaniumBackup but it would import the contacts properly. I pulled the contacts2.db from the backup file and used your script to get the data out.
I did have to make some changes to the mimetypes in the SQLs but apart from that everything worked perfectly. I think the mimetypes may have changed between android versions.
The modified portion of the script:
Code:
# Get the First_Name and Last_Name in one query:
# NOTE: Need to remove occasional CTRL-M's with sed.
NAME=`$CMD -csv $CONTACTSDB "SELECT data2, data3 FROM data WHERE raw_contact_id=$RAWID AND mimetype_id=5;" | sed ':a;N;$!ba;s/\n//g'`
# Get the Email address:
# NOTE: This will simply append multiple email adresses to one long string, with ";" separator!
EMAIL=`$CMD $CONTACTSDB "SELECT data1 FROM data WHERE raw_contact_id=$RAWID AND mimetype_id=1;" | sed ':a;N;$!ba;s/\n/;/g'`
# Here we get the "Home", "Mobile", Work" and "Other" (labeled) phone numbers.
# NOTE: Each of these may return multiple responses, so we need to convert new-lines (\n) to ",".
HPHONE=`$CMD $CONTACTSDB "SELECT data1 FROM data WHERE raw_contact_id=$RAWID AND mimetype_id=6 AND data2=1;" | sed ':a;N;$!ba;s/\n/,/g'`;
MPHONE=`$CMD $CONTACTSDB "SELECT data1 FROM data WHERE raw_contact_id=$RAWID AND mimetype_id=6 AND data2=2;" | sed ':a;N;$!ba;s/\n/,/g'`;
WPHONE=`$CMD $CONTACTSDB "SELECT data1 FROM data WHERE raw_contact_id=$RAWID AND mimetype_id=6 AND data2=3;" | sed ':a;N;$!ba;s/\n/,/g'`;
OPHONE=`$CMD $CONTACTSDB "SELECT data1 FROM data WHERE raw_contact_id=$RAWID AND mimetype_id=6 AND data2=7;" | sed ':a;N;$!ba;s/\n/,/g'`;
PHONES="$MPHONE,$HPHONE,$WPHONE,$OPHONE";
Thanks so much for your post.
I can't thank you enough for this information!
My S2 died on me a month back, and the only backup of my contacts (which are not synced on Gmail) were an old TB backup from April 2012. At least thanks to this knowledge I know what and where to look for!
Thanks a lot
Help request with command
Hello.
I am trying to convert a recovered contacts database from contacts2.db to Google compatible cvs file with sqlite3.exe.
The recovered database came from a Samsung N7000 and it is composed as follow in the table data:
SELECT "_id", "package_id", "mimetype_id", "raw_contact_id", "is_primary", "is_super_primary", "data_version", "data1", "data2", "data3", "data4", "data5", "data6", "data7", "data8", "data9", "data10", "data11", "data12", "data13", "data14", "data15", "data_sync1", "data_sync2", "data_sync3", "data_sync4", "is_read_only" FROM "data";
There are 2 lines for a single contact. mimetype_id 6 is the line of the name in data1; mimetype_id 5 is the line of the number in data1
Is there a command to properly export that database? I have no skills in programming and scripting.
I have to complete an import of a database to a new android phone from a N7000 old contacts2.db :crying:
Just to say that the code
Code:
SELECT "_id", "package_id", "mimetype_id", "raw_contact_id", "is_primary", "is_super_primary", "data_version", "data1", "data2", "data3", "data4", "data5", "data6", "data7", "data8", "data9", "data10", "data11", "data12", "data13", "data14", "data15", "data_sync1", "data_sync2", "data_sync3", "data_sync4", "is_read_only" FROM "data";
gives something like that:
5406,,5,2335,0,0,0,349563erased for privacy,2,
5407,,6,2335,0,0,0,"federico agronomo",federico,agronomo
And so on...
Since I have no skills in developing and scripting, is there anybody how can help me extracting universal VCF from that N7000 sql contacts2.db specific database? I can send it to you by eMail if you write to me in pvt. I'll apprecciated.
I did a lot of tests, but I can not create a proper query or run a proper script.
I was able only to extract the database in a way like the one mentioned. I have quite 1000 contacts :crying:
Thank you in any way for help.
Kind regards.
THREAD CLOSED!
There are now hundreds of contacts & backup Apps on Google Play, so I'm closing this old thread.
But here are a few free ones that I really like:
Contact Backup & Restore Gmail
Super Backup : SMS & Contacts
MCBackup - My Contacts Backup
They should be able to export to CSV, VCF and XML files...
ENJOY!
Hello Gentlemen. Sorry to disturb you with a noob question but I cannot solve it for the life of me and I have tried everything I could find on the forums. I have an Aluratek Cinepad AT107F. I have successfully rooted it. I have full super user permissions and I have no problems deleting any files or folders EXCEPT one directory. Let me explain the situation.
I recently did a firmware update and it included brand new APKs for Youtube, also added Google Play Support, and added Angry Birds.
I am unable to update Youtube to the latest version. It gives a "Package file was not signed correctly. Uninstall the previous copy of the app and try again.".
So I attempted to do just what it had asked. I rooted just to do this. I tried removing it with Titanium Backup, I tried Root Uninstaller, I tried Root Explorer, I tried deleting using the "adb shell rm" command. I still get a "Read-only file system".
Even though I have root and granted root access to Root Explorer, I am unable to change the permissions for this /oem/apps/ directory. It's on the top level of the internal memory. I have no external SD card.
I've spent 10+ hours trying to figure this out and I'm sure someone knows something that I don't and can fix this super easy. I'm asking for your help, you're my only hope!
Your issue is that the update gave you modded files that the OEM doesn't want deleted. Most likely do to them not have permission from Google to do so. You will be hard pressed to find help here as we mainly deal with official releases and apks.
Wayne Tech S-III
zelendel said:
Your issue is that the update gave you modded files that the OEM doesn't want deleted. Most likely do to them not have permission from Google to do so. You will be hard pressed to find help here as we mainly deal with official releases and apks.
Wayne Tech S-III
Click to expand...
Click to collapse
Am I in the wrong sub forum? Is there another forum at XDA-Developers that could help?
Vindicoth said:
Am I in the wrong sub forum? Is there another forum at XDA-Developers that could help?
Click to expand...
Click to collapse
No nowhere on the site really deals with unofficial android versions and knock of devices. Your best bet will be the OEM
Read the error message again. The file is on a read-only filesystem. That is why you cannot delete it. In other words, you lack permission to write to the partition.
System partitions get mounted readonly to prevent modification. To remount /system as read-write,
Adb shell mount -o remount,rw /system
If you get "not permitted" error, your ROM's ro.secure means you cannot execute mount operations passed with adb command. So instead you must first open the shell,
> AdB shell
# mount -o remount,rw /system
Now you can delete:
Adb shell rm /system/file
If it is a directory
Adb shell rm -rf /system/dir
If you accidentally mistype that last command with a space beyween that first forward-slash and "s" you will have a very unworkable device....and that's why it is mounted read-only.
If the file is on a different read only filesystem, identify the partition the file is on and
Adb shell mount -o remount,rw /dev/block/id /LocToMountTo
You may have to specify the type
-t fstype
Adb shell mount
will tell you this
Don't forget to remount it as read-only (ro) when you are done
anotherguy19 said:
Read the error message again. The file is on a read-only filesystem. That is why you cannot delete it. In other words, you lack permission to write to the partition.
System partitions get mounted readonly to prevent modification. To remount /system as read-write,
Adb shell mount -o remount,rw /system
Now you can delete:
Adb shell rm /system/file
If it is a dir
Adb shell rm -rf /system/file
If you accidentally mistype that last command with a space beyween that first forward-slash and "s" you will have a very unworkable device....and that's why it is mounted read-only.
If the file is on a different read only filesystem, identify the partition the file is on and
Adb shell mount -o remount,rw /partition/id /folder
You may have to specify the type
-t fstype
Adb shell mount
will tell you this
Don't forget to remount it as read-only (ro) when you are done
Click to expand...
Click to collapse
Well I can access /system just fine. The problem is the /oem/apps folder is not in the /system folder. It's in the top level folder.
/system and /oem/apps are in the root directory. I can change the permissions on /system just fine using Root Explorer, but when trying to change the permissions using any method possible, /oem wont change.
This happens alot on Devices out of China and other places that sell knock offs. They make it so you cant delete their apps and if you do then it bootloops which only a reflash will fix.
Vindicoth said:
Well I can access /system just fine. The problem is the /oem/apps folder is not in the /system folder. It's in the top level folder.
/system and /oem/apps are in the root directory. I can change the permissions on /system just fine using Root Explorer, but when trying to change the permissions using any method possible, /oem wont change.
Click to expand...
Click to collapse
/system and /oem are different partitions then
mount | grep oem
or
adb shell mount | grep oem
Will tell you what /dev/block/xxxx the /oem is on and if it is mounted as read-only (ro)
> adb shell
# mount -o rw,remount /oem
zelendel said:
This happens alot on Devices out of China and other places that sell knock offs. They make it so you cant delete their apps and if you do then it bootloops which only a reflash will fix.
Click to expand...
Click to collapse
Ever get the feeling that you're writing in invisible ink lol?.. Eventually he will listen! You have great patience!
zelendel said:
This happens alot on Devices out of China and other places that sell knock offs. They make it so you cant delete their apps and if you do then it bootloops which only a reflash will fix.
Click to expand...
Click to collapse
Referencing zelendel's post, the device could fail to boot because it is looking for the removed app in /oem
So you could try to remove it from being referenced by the startup scripts.
Try doing a search for files that could be referencing the apk you want to remove.
> adb shell
# grep -r AppName.apk /system/etc
CtrlAltDelIrl said:
Ever get the feeling that you're writing in invisible ink lol?.. Eventually he will listen! You have great patience!
Click to expand...
Click to collapse
If it bootloops then I'll just flash it again, but its worth a try isn't it? Thanks anyways.
anotherguy19 said:
Referencing zelendel's post, the device could fail to boot because it is looking for the removed app in /oem
So you could try to remove it from being referenced by the startup scripts.
Try doing a search for files that could be referencing the apk you want to remove.
> adb shell
# grep -r AppName.apk /system/etc
Click to expand...
Click to collapse
Thank you for your very helpful posts and willingness to help me possibly bootloop my device
when I type the grep command it says it is not found, so I will download busybox onto my tablet and try those commands.
Vindicoth said:
Thank you for your very helpful posts and willingness to help me possibly bootloop my device
when I type the grep command it says it is not found, so I will download busybox onto my tablet and try those commands.
Click to expand...
Click to collapse
So after running mount | grep oem it shows this
[email protected]:/ # mount | grep oem
/dev/block/nandi /oem cramfs ro,relatime 0 0
So is it possible to change this directory to rw? I tried the mount -o rw,remount oem but it doesnt show anything after i input the command
[EDIT]
So it seems that the cramfs file system is read-only.
Vindicoth said:
Thank you for your very helpful posts and willingness to help me possibly bootloop my device
when I type the grep command it says it is not found, so I will download busybox onto my tablet and try those commands.
Click to expand...
Click to collapse
I love breaking things. It's only then we can figure out how it works.
Busybox is exactly what you need.
grep will search for strings within all files and subdirs of /system/etc; matching whatever you type for "AppName.apk"
It is a case-sensitive search unless you add
-i
So
grep -ri ....
or
grep -r -i....
Both would work.
But you don't even know if you need to be searching for a file to modify.
You may want to go ahead and delete the file and reboot.
If it fails, note the file(s) you deleted and know the system is looking for them and then proceed with seeing if you can identify a file in /system/etc that is looking for it's presence.
---------- Post added at 11:59 PM ---------- Previous post was at 11:53 PM ----------
Vindicoth said:
So after running mount | grep oem it shows this
[email protected]:/ # mount | grep oem
/dev/block/nandi /oem cramfs ro,relatime 0 0
So is it possible to change this directory to rw? I tried the mount -o rw,remount oem but it doesnt show anything after i input the command
[EDIT]
So it seems that the cramfs file system is read-only.
Click to expand...
Click to collapse
Commands that complete "quietly" mean they were successful.
If you type
# mount | grep oem
After
# mount -o rw,remount /oem
/dev/block/nandi /oem cramfs ro,relatime 0 0
Should have changed to
/dev/block/nandi /oem cramfs rw,relatime 0 0
anotherguy19 said:
I love breaking things. It's only then we can figure out how it works.
Busybox is exactly what you need.
grep will search for strings within all files and subdirs of /system/etc; matching whatever you type for "AppName.apk"
It is a case-sensitive search unless you add
-i
So
grep -ri ....
or
grep -r -i....
Both would work.
But you don't even know if you need to be searching for a file to modify.
You may want to go ahead and delete the file and reboot.
If it fails, note the file(s) you deleted and know the system is looking for them and then proceed with seeing if you can identify a file in /system/etc that is looking for it's presence.
---------- Post added at 11:59 PM ---------- Previous post was at 11:53 PM ----------
Commands that complete "quietly" mean they were successful.
If you type
# mount | grep oem
After
# mount -o rw,remount /oem
/dev/block/nandi /oem cramfs ro,relatime 0 0
Should have changed to
/dev/block/nandi /oem cramfs rw,relatime 0 0
Click to expand...
Click to collapse
Followed those steps and its still
/dev/block/nandi /oem cramfs ro,relatime 0 0
Apparently the cramfs is a read only filesystem by design.
Vindicoth said:
Followed those steps and its still
/dev/block/nandi /oem cramfs ro,relatime 0 0
Apparently the cramfs is a read only filesystem by design.
Click to expand...
Click to collapse
You need to create an image of the partition, mount the partition on a system with tools to edit it, extract the contents, and re-create the cramfs.
You would need a linux box or linux virtual machine like Virtualbox, or maybe cygwin has the tools. I would just download and burn a Linux LiveISO and run it from VirtualBox, creating a shared folder to get access to the image file (oem partition).
With debian system, you would do
# apt-get install cramfsprogs fusecram
fusecram allow you to mount the cramfs partition on Linux PC via
# mount -t loop cramfsOEM.partition /mnt/workingdir
However since we cannot simply mount the filesystem on the device by plugging into the usb port of the linux machine and mounting from there, we must first create an image (file) of /dev/block/nandi.
> adb shell
# dd if=/dev/block/nandi of=/nandi.img bs=4k
And then copy it to our pc so we can work with the file.
# exit
> adb pull /nandi.orig.img .
Now we can transfer this file to a machine with the requisite cramfs tools to modify the file.
If you look back, I wrote
# mount -t loop cramfsOEM.partition /mnt/workingdir
replace cramfsOEM.partition for nandi.orig.img, or whatever you named it.
Workingdir needs to exist, so
# mkdir /mnt/workingdir
Now you will have to look up cramfsprogs which will allow you to extract the contents to modify. On the Debian or Ubuntu linux machine "/mnt/workingdir" would be the equivalent of "/oem" on your Android device.
However, all this is could very well be for naught, as it is likely the firmware has marked this partition as "signed" so if we try to write back our modified image, the system will fail to boot since the size will be different. On the other hand, the firmware may very well just check to see the partition size is correct. And since you are decreasing the size, the new cramfs image created with a linux box will (should) fit in the old partition.
Fyi, an image file is like a zip file without the compression.
After you modify the cramfs, you can write it back with something like
> adb push cramfs.mod.img /
> adb shell
Then write over the old partition. However, you shouldn't write over a mounted file system so
# umount /dev/block/nandi
Then write over it.
# dd if=/cramfs.mod.img of=/dev/block/nandi bs=4k
Then reboot
# shutdown -r now
And see what happens.
anotherguy19 said:
You need to create an image of the partition, mount the partition on a system with tools to edit it, extract the contents, and re-create the cramfs.
You would need a linux box or linux virtual machine like Virtualbox, or maybe cygwin has the tools. I would just download and burn a Linux LiveISO and run it from VirtualBox, creating a shared folder to get access to the image file (oem partition).
With debian system, you would do
# apt-get install cramfsprogs fusecram
fusecram allow you to mount the cramfs partition on Linux PC via
# mount -t loop cramfsOEM.partition /mnt/workingdir
However since we cannot simply mount the filesystem on the device by plugging into the usb port of the linux machine and mounting from there, we must first create an image (file) of /dev/block/nandi.
> adb shell
# dd if=/dev/block/nandi of=/nandi.img bs=4k
And then copy it to our pc so we can work with the file.
# exit
> adb pull /nandi.orig.img .
Now we can transfer this file to a machine with the requisite cramfs tools to modify the file.
If you look back, I wrote
# mount -t loop cramfsOEM.partition /mnt/workingdir
replace cramfsOEM.partition for nandi.orig.img, or whatever you named it.
Workingdir needs to exist, so
# mkdir /mnt/workingdir
Now you will have to look up cramfsprogs which will allow you to extract the contents to modify. On the Debian or Ubuntu linux machine "/mnt/workingdir" would be the equivalent of "/oem" on your Android device.
However, all this is could very well be for naught, as it is likely the firmware has marked this partition as "signed" so if we try to write back our modified image, the system will fail to boot since the size will be different. On the other hand, the firmware may very well just check to see the partition size is correct. And since you are decreasing the size, the new cramfs image created with a linux box will (should) fit in the old partition.
Fyi, an image file is like a zip file without the compression.
After you modify the cramfs, you can write it back with something like
> adb push cramfs.mod.img /
> adb shell
Then write over the old partition. However, you shouldn't write over a mounted file system so
# umount /dev/block/nandi
Then write over it.
# dd if=/cramfs.mod.img of=/dev/block/nandi bs=4k
Then reboot
# shutdown -r now
And see what happens.
Click to expand...
Click to collapse
Wow that was very detailed. I thought I might have to do something like that. Thanks so much again. I'll try this in the morning since it's getting very late here. I'll go ahead and download a linux livecd tonight.
Well the script should be working, it is 'ettin.sh' now. To get VNC to work on separate device just change 'eth0' to 'wlan0' or whatever other interface you know how to configure, like usb maybe, in the 'init.sh' script that resides in ~/ on the images you can download from http://linuxonandroid.org/downloads/. Cool. If anyone can help with the usb configuring that would be cool.
Also on the different roms your storage might be different so just change that to what suites your environment,
BAD NEWS, After my first launch and exiting the VNC on my laptop (Ubuntu) My phones home button and a few other functions stopped working, a reboot remounts the root and system partitions and it goes back to normal essentially, I haven't noticed any adverse affects after the reboot.
Good luck.
EDIT: If you can't help me, you probably shouldn't attempt any of this.
I want to connect to backtrack over usb from Ubuntu on my laptop instead of from my phone. Is there a good way of doing this?
The thing is, I didn't chroot, I mounted the image, copied the contents of '/etc/*' and '/sbin/*' to the '*/etc' '*/sbin' dirs of the mounted image. and then mounted it's (the image's) contents to Androids rootfs '/' then I set up a little script to set up the environment and execute '/bin/bash -i init.sh' and again reset PATH to include /system/*.
Then in the init.sh script I want to set up a way to 'vnc' through' usb to backt4acks desktop. How would I do this?
EDIT: I started this script blelow, its not complete, feel free to use it however you like.
If anyone is good with bash I need help with loops in Bash or shell, ksh, or ash. So if you change it at all share those changes here. I haven't tested any of the loops, I wrote this here and need to do some reading on shell scripting, peace out.
Just NOTE that I would not execute this script yet, it is NOT YET COMPLETE.
I will be making minor changes as I learn and test things out, Also this was inspired by the autobootscrip (sp?) that the Linux on Android Installer app uses.
mintberrycrunch revision
I need a little help with the mounting part, "is there anybody out there?"
EDIT: Maybe escape '\' characters...
Maybe this will work? 'if [ ! -d "/$d" -a "$d" \!= "dev" -o "$d \!= "mnt" ]; then ' I want it to test whether the directory does not already exists in / and if it is not named dev or named mnt then mount it. Do you think this is a decent way to do it? Any suggestions?
Sorry, I still haven't tested this script yet, so if you brave and see any changes that will make it work I am not resposible. But it is getting close to when I will run it for the first time. The difference between this and the one I wrote for Kali is this should be a one stop shop so to speak for getting into a non chroot enviroment where you can access Bt's tools from Android directly. Apperently about a year ago someone was trying something similar called "debian installer" or something, I haven't tried it out so maybe that actually works, idk, I am happy with this.
LOL
LMFAO!
I just realized I don't think I needed to worry about testing if the variable $d was named 'dev' or 'mnt' because they should get skipped by already being mounted. If not I guess if it works as is, may be extra precaution.
Done! Almost ;{)-
Uhmm.... hope you can see the screenshot...
its over wifi though, I had to edit the init.sh script just eth0 to wlan0 in 2 lines, duh! where is eth0 on phone?
Check out the filesystem
Bada bing, bada doom, dare ya goes`a for ya.
V-0.021
This should support args, if your using different image or location such as external sdcard or want to mount to a different location
Notes: 'ettin.sh' [/IMAGE/LOCATION] [IMAGENAME] [MOUNT/POINT]'
1) Don't add a trailing forward slash for image location, the first argument. Doing so may conflict with the script.
2) Don't add suffix to image name ( second argument), the script uses a wild card period wild card syntax to account for .iso, .raw, or .img aswell as version numbers, so if its backtrack-v-r5.img all you need is 'backtrack'.
3) Default mount is "/data/local/mnt"
for example:
Code:
ettin.sh /storage/extSdCard/ubuntufolder ubuntu /mnt/myLinux
That ^^^ will use a ubuntu image on the external sdcard and mount it to a folder called "myLinux" within the "/mnt" directory.
The script below...
Code:
#!/system/bin/sh
# ettin.sh v-0.021, "Two heads are better than one."
# Written by 'Edge-Case' @ 'forum.xda-developers.com'
# This version is attempting to use symbolic linking to correct some issues.
## The purpose of ettin.sh is to merge a Linux system with Android
## hopefully allowing both systems to run in synch with each other.
## It is probably best to be ran on devices with multi-core processors.
##"""Notes on Backtrack and Kali Linux Distributions:
##"""1) Postgresql needs to be configured in order for metasploit to connect to
##""" the database. This might be possible within the script. Perhaps
##""" it is simply a matter of configuring SSH? I don't know.
##"""2) The Kernel needs to be patched to allow for packet injection and
##""" monitor mode for an external 'wi-fi' card as the factory hardware
##""" does not support monitor mode, at least on Samsung Galaxy SIII.
#########
# Prep ##
#########
mount -wo remount systemfs /system
mount -wo remount rootfs /
imloc=${1:-"/storage/sdcard0/backtrack"}
imname=${2:-"backtrack"}
subset=${3:-"/data/local/mnt"}
#Check for root!
if [ ! -d "$subset" ]; then mkdir "$subset"; fi
busybox mount -wo loop ${imloc}*/${imname}*.* ${subset}
check_mnt="`echo $?`"
if [ ${check_mnt} != 0 ]; then echo "Something is wrong with mounting, check the situation!" && exit; fi
##################################################
# Copy contents of special directories to image ##
##################################################
if [ ! -f "${imloc}/success.txt" ] ; then cp -ai /sbin/* ${subset}/sbin && echo "Files have been copied on `date`." > ${imloc}/success.txt; fi
if [ -e "/root/*" ]; then cp ai /root/* ${subset}/root; fi
######################################################
# Make directories needed and mount the file-system ##
######################################################
cd ${subset}
for d in `ls` ; do if [ ! -d "/${d}" ] ; then mkdir /${d} && busybox mount --rbind ${subset}/${d} /${d} ; fi ; done
busybox mount --rbind ${subset}/root /root
busybox mount --rbind ${subset}/sbin /sbin
busybox cp -rspi ${subset}/etc/* /etc # We'll try making symbolic links from the image to Android's /etc instead.
# Add any directories as needed.
###############################
# be safe before we continue ##
###############################
mount -ro remount systemfs /system
mount -ro remount rootfs /
######################################
# continue building the environment ##
######################################
export PATH="/system/bin:/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin:/system/xbin:/usr/games"
export TERM=linux
export HOME="/root"
#################################
# further down the rabbit hole ##
#################################
/bin/bash -i ~/init.sh
Pretty happy with it now, so far. Touch FIXED
Well, I am tethered from my phone to my laptop, running ubuntu, this is how I am on the internet, so my phone is doing that...
I ran 'ettin.sh /storage/sdcard0/ubuntu ubuntu' using V-0.022 and only had two files that needed overwriting.
The touch is working....check
adb is working...check
Everything is mounted ...check
Apps like browser and dsploit are working on the phone as usual...check
'apt-get update && apt-get upgrade'....check (over adb, no chroot, as [email protected] using bin/bash)
'which nmap' .....check ("
[email protected]:~# which nmap
/usr/bin/nmap
")
Well **** ya'll looks like this might work... I am now going to consider modifying a stock rom to support users and login that meshes with debian and its "shadow" file etc etc, also I need to work out permissions and enviroment variables...
I'm happy...
Also for pentesting distro's "modifying Android kernel" tutorials would be nice, anybody able to point me to one?
Hi,
Can't really find a tutorial for our phone, so I'll be, well, making a tutorial
Some notes:
- Compiling is really slow, VERY. Probably RasPi speed.
- Putting the chroot in a specific location helps, due to our phone having weird mountpoints (try typing `mount` and see what happens)
- You can ADB shell into the phone, type "stop" so Zygote is stopped (less RAM usage, less CPU usage, basically pure Linux is running)
- ^ Remember to type "start" after finished playing with the chroot, or you get a frozen phone until forced reboot!
You may be able to adapt this guide to other distros too (although unlikely)
REQUIREMENTS
- Enough juice for playing games for a long time
- A WORKING busybox with "tar xvjpf" available
- You are rooted
1. First of all, su and mount root read/write.
Code:
su; mount -o remount,rw /
2. cd into /data/local
Code:
cd /data/local
3. mkdir linux (or other names if you prefer)
Code:
mkdir linux
4. cd into it
Code:
cd linux
5. Download a stage3 tarball USE ARMv7l, our phone isn't hard-floated and put it in current directory.
- If you use an external browser, make sure the permissions isn't set to r--------, chmod 755 to be sure
- If you use wget, you probably don't have to worry about permissions
6. Extract it. Replace blahblahblah with your file name
Code:
tar xvjpf blahblahblah.tar.bz2
7. Make a chroot script which contains all the necessary bind mounts and commands. Make sure it is in the same dir as your chroot!
Code:
#!/system/bin/sh
# Enable network forwarding to the chroot
sysctl -w net.ipv4.ip_forward=1
# Mount proc to "current directory"/proc
mount -t proc proc proc/
# Mount sys, bind
mount -o bind /sys sys/
# Mount dev, bind
mount -o bind /dev dev/
# Symlink proc/self/fd to dev/fd, or portage will fail
ln -s proc/self/fd dev/
# Write resolvconf
echo "8.8.8.8" > etc/resolv.conf
# Make sure the permissions are correct
chmod 755 etc/resolv.conf
# Chroot using /bin/bash as shell
chroot . /bin/bash
Then chmod 755 the script
Code:
chmod 755 chroot.sh
8. All Gentoo users should know this...
Code:
source /etc/profile
9. Edit /etc/portage/make.conf
Code:
nano /etc/portage/make.conf
10. Add these lines to it
Code:
# This will disable userfetch, making name resolution work.
FEATURES="-userfetch"
11. Adjust USE flags and other configs as necessary... save it
12. Enjoy Gentoo on your phone! :laugh::laugh:
13. Just Ctrl+D to exit chroot