Activate HCE in Bliss-x86 (Android-x86) - Android Q&A, Help & Troubleshooting

Hello! I can't write in Soft Development section, so I'm writing here.
I need to make a build of any Android-x86 with HCE ability. I tried to modify common Android-x86 and CM builds but did not succeed. Now I'm experimenting with Bliss-x86. The problem is that even when I define NFC service to be built into the image it doesn't appear in the OS.
So, what I have now:
- "external/libnfc-nci", "external/libnfc-nxp" and "packages/apps/Nfc" packages are defined as default in manifest.xml
- PRODUCT_PACKAGES parameter includes "Nfc" string in device/generic/common/device.mk and in packages.mk
- android.hardware.nfc.xml and android.hardware.nfc.hce.xml files are to be copied into the image
When I resync the android tree, I see that NFC related files are fetched from the server. But after the building I don't see /system/apps/Nfc.apk file and /data/data/com.android.nfc folder. NFC service is not starting on boot. I assume it's not compiled at all. Why that happens? The build process goes OK.
What exactly should I do to enable Nfcservice in my build?

Well, I solved the issue by defining "NfcNci" in device.mk (not "Nfc"). There also may be additional packets to include, e.g. "libnfc-nci", "libnfc-nxp", "libnfc_nci_jni", "Tag", "com.android.nfc_extras".

Related

[Q][SOLVED] Kernel Modules: Built-In -> External?

Last night, I built my first Android kernel from source (opensource.samsung.com) - it was a great experience. However, I wasn't able to accomplish my goals.
I'm trying to build a few specific kernel modules, but I need them to be external modules (*.ko). The build works well, but from what I can tell by the modules.builtin file(s), the specific ones I'm trying to get out of this are built-in only. During the compile, it generates *.o files just fine. At the end of the compile, it shows it used MODPOST to create 17 .ko files. All of these are noted in the modules.order file(s).
It seems that no matter what I do, I can't change whether these are built-in or external. I tried modifying the modules.builtin and modules.order files (there's multiple copies of these that reference the specific modules I want), but these just seem to get overwritten at build time.
I've done lots of searching here and abroad, and I can't seem to find the magic I'm looking for. I've tried many things, most notably:
- make <path>/<module>.ko
- make modules
- other various commands
- cd into subdir that contains the modules I want, running make there (no targets)
Doing more research, it seems it might point to the Makefile that's in the subdirectory of the modules I want. I see various "hello world" Makefiles out there, but most of these don't seem to be completely relevant to what I'm trying to do - plus I already have a Makefile for these modules.
I'm assuming that what I probably need to do is modify the Makefile that's specific to these modules, to force it to make .ko files instead of .o files. I'm also assuming that I can't simply change the lines in the Makefile to read "<module>.ko" instead of "<module>.o". I realize that there's a difference in these files.
I also thought the answers might lie in looking at the Makefile of one of the other 17 external modules that get created. However, it appears these are no different than the Makefile of the modules I'm after.
Okay, enough rambling - I'll give some details now, in case they are needed. I've downloaded the I337 source (MF3) from opensource.samsung.com. The three modules I need (so far) are: msm_kgsl_core.o, msm_adreno.o, msm_z180.o. The makefile that includes these three came from <source-root>/drivers/gpu/msm, and it is attached. I may need more than these three files later, but this is a good start.
I don't want to hear a bajillion questions about, "What are you planning to do with these files?" or "Why do you need them anyway?" - the answer is a longer story than this thread. Trust me, I just need these.
So what do I need to do to make these three files? Thank you for your help.
EDIT/SOLUTION:
obj-y = built-in module
obj-m = external module
I still can't track down the source of the three variables in the last three lines of this Makefile, but they are obviously returning "y". I changed the last three lines manually to:
Code:
obj-m += msm_kgsl_core.o
obj-m += msm_adreno.o
obj-m += msm_z180.o
... and then ran a "make clean" and "make modules" again. Voilà: 20 modules built by MODPOST.

[Q] What file level function is used to read/write SQLite database file

Hi,
I am working on some sort of implementation for SQLite file encryption in Android devices. I would like to get some info about the low level function inside the SQLite source code (Amalgamation version) that is responsible for writing the database file to disk/storage (or reading from it)?
Based on my research inside SQLite source code (Android version), I found different functions that might be related to disk read/write operations. These functions are: winWrite(), unixWrite(), seekAndWrite(), unixRead().
So my question is that which one of these functions (or other lower functions) is responsible for the final writing to storage, right after the database file is closed and need to be saved?
Knowing that specific function can help us embed our encryption test module in the proper place of SQLite so that the encryption will occur exactly before saving database file to disk.
Thank you for any help.
P. S. I am not interested in current SQLite encryption products (Sqlcipher, SEE, etc) and prefer to test my own encryption algorithm.

Paranoid Networking?

I have been having issues with the Paranoid Networking feature added to the Linux Kernel preventing some of my programs from creating sockets.
Appearently the process must be of a certain group id, either one of the AID_INET or AID_BT groups in the 3000's range I believe. I was cgreping through the source code and it seems a few calls to "in_egroup_p(gid_t)" are made to test if the process's group is 'correct' when ANDROID_PARANOID_NETWORKING is defined. If I understood the "in_egroup_p()" function, it should return 1 (retval) if there is a match. so what I did was this in the c file for inet in the kernel. I hope the kernel gets compiled, I just did a "make clean" and "brunch d2spr" it is cm-10.1
Code:
#ifdef ANDROID_PARANOID_NETWORKING // this was there
if ( in_egroup_p(0) == 1) {
return 1;
} // I added this
return in_egroup_p(AID_INET) || capable(I_FORGOT_HERE); //this was there
And I am hoping that when I execute the program as root, it will have permission to create a socket. Also note, that the programs I am having problems for are glibc based that I copied and mounted from a .img file. Programs such as postresql, gdm3, and others. It is a debian distro that I mount with an init service when a property is set to 1.
If this does not work, I will need to run without the Paranoid Networking patch to the kernel. What config file do I need to edit so that ANDROID_PARANOID_NETWORKING does not get defined or built into the kernel?
edit: I feel like this, " IM ROOT B****, STAY OUTTA MY WAY! " And then that scene from Jurrasic Park comes to mind
http://www.youtube.com/watch?v=RfiQYRn7fBg
Still no go
Code:
[email protected]:/etc/init.d# ./postgresql start
[....] Starting PostgreSQL 9.1 database server: main[....] The PostgreSQL server failed to start. Please check the log output: 2014-01-06 16:48:59 UTC LOG: could not create IPv6 socket: Permission denied 2014-01-06 16:48:59 UTC LOG: could not create IPv4 socket: Permission denied 2014-01-06 16:48:59[FAILWARNING: could not create listen socket for "localhost" 2014-01-06 16:48:59 UTC FATAL: could not create any TCP/IP sockets ... failed!
failed!
*BEEP* *BEEP* *BEEEEEEEEEEP*
Hmmm... Maybe the kernel didn't compile? I dunno . ...
Where is the config file for the kernel that comes with CM-10.1 so I can turn off this paranoid networking?
My philosophy is that they shouldn't be there to start a socket in the first place, but if I want to, I should be able to. This is where I need something akin to sudo, but as root this is pissing me off, let me create a socket por favor!
I tried a lot of things but this just isn't working right. Some one please, help.
I've about had it up to my neck with Android. ****ing bull ****, I didn't know I was buying a bad ass phone with Chains and Iron Bars for an OS. FREEEEEEEDOOOOOOMMMMMNNNNNNNNN
After breaking my build, I deleted my kernel directory tree and re-sync'ed. I then did "source build/envsetup.sh" and "breakfast d2spr" and then I when to "kernel/samsung/d2/" and did "make menuconfig" then I unselected in networking options "Only specific groups can create sockets" or some such option. and then "brunch d2spr". We shall see if that works.
Update:
Did not work. Said unfinished jobs yada yada
So what I am trying now.
I cleaned everything, bummer!
I copied my kernel dir to another location
I pulled my kernel config with adb
I used that with 'make menuconfig' and edited out the 'only allow certain groups' bit
I saved that
copied it to my source in kernel/samsung/d2/
I edited AndroidKernel.mk to use that config file instead ( the assignment occurs at the top of the file)
croot
brunch d2spr
waiting till tomorrow more than likely....
Update: ( 3 hours later )
Its taking long enough
seems to be working
maybe I can make a config in my source tree and then make mroper and clean, aswell as in the KERNEL_OBJ dir in out/*, then brunch. to avoid needing two kernel source trees and some confusion in case I modify any of the source and not just the config.
AndroidKernel.mk specifies which config file to use instead of the other pre-configured files, just start by pulling a config off your phone with the same rom and device your building for.
Code:
adb pull /proc/config.gz
That will pull the conpressed file off your phone to the current working directory, then gunzip it. Then launch
Code:
make menuconfig
from your kernels root dir, for me that is "/home/$USER/android/system/kernel/samsung/d2/"
And then once the menu is up, use the arrows to scroll down to the bottom and use enter or return to select the option to use a different configuration file, it should change menus to one with ".config" erase that and change it to point to your config file, I just copied mine to the same directory mentioned above with the name "config" the same without the "." (dot/period) in front.
After that, from the same dir ( as well as in the KERNEL_OBJ/ dir somewhere within the out/ dir)
Code:
make mrproper && make clean
to clean the previous kernel make.
After all of that, you can 'croot' and 'brunch $YOUR_MODEL', but not before making sure you change which config file to use in your kernel's root dir in it's AndroidKernel.mk file. Its at the top, I used an absolute path for my installation, just in case.

[REFERENCE] My experience building the Tucana kernel & Rom

Little info on what this all is.
If you are expecting a working kernel or rom after reading this all. That is not what this thread is.
What this all is intended for is material to help with getting you to the next step, if you are at a road block.
What will you gain from all this?
A fair bit of knowledge on working with msm-4.14 kernel and maybe others.
Make sure to have your search bar ready with a piece of the issue (key word of the error will do) and go through the hidden tabs until you pick up your error, with the browser search function (ctrl+f on most browsers)
Spoiler: OLD info
Method I used for building the kernel
1. Dowload AOSP common-kernel-4.14 through git
OR
ANDROID 10 Specific
refs/heads/q-common-android-4.14 - kernel/manifest - Git at Google
2. Download Xiaomi TUCANA-Q-SOURCE for android 10 source code from MiCode git.
ALL VARIANTS
GitHub - MiCode/Xiaomi_Kernel_OpenSource: Xiaomi Mobile Phone Kernel OpenSource
Xiaomi Mobile Phone Kernel OpenSource. Contribute to MiCode/Xiaomi_Kernel_OpenSource development by creating an account on GitHub.
github.com
**OR**
TUCANA Android 10
GitHub - MiCode/Xiaomi_Kernel_OpenSource at tucana-q-oss
Xiaomi Mobile Phone Kernel OpenSource. Contribute to MiCode/Xiaomi_Kernel_OpenSource development by creating an account on GitHub.
github.com
3. extract if you did not use git to download sources.
You will need AOSP 10 R29 downloaded. R29 Matches Stock MIUI 12.0.4
4. mkdir kernelbuild (can be what ever) then extract both AOSP KERNEL (git) ,build, common, kernel, prebuilts, master-prebuilts folders to the kernelbuild folder.
5. just use TUCANA source and don't merge with AOSP common source. Don't delete common either
4. copy AOSP kernel folder (contains 4.14) to TUCANA source and merge (Still not sure if you need to do this)
5. make sure you have AOSP-10 from git already. If not, repo it.
6. Go to AOSP-10 REPO>prebuilts>gcc>linux-x86>aarch64 location and get (aarch64-linux-android-4.9) folder and copy it to root Tucana source files folder make a directory call toolchain TUCANA source. in prebuilts>gcc>linux-x86>aarch64 folder replace and merge all
7. Not sure if this is outdated but got the export info from MSM section at https://github.com/MiCode/Xiaomi_Kernel_OpenSource/wiki/How-to-compile-kernel-standalone
e.g.
export CROSS_COMPILE=/<toolchain-location/prebuilts>GCC>/bin/aarch64-linux-android-
mkdir out
cd out
Taken from mi code section for MSM 4.14
export ARCH=arm64
export SUBARCH=arm64
export DTC_EXT=dtc
Set CONFIG_BUILD_ARM64_DT_OVERLAY=y
(does not work from what i can tell. Have to enable using menuconfig)
if not inside the "out" folder use O=out on next command, will also have to type cd ../ to go back in source if using this command.
(-jN (N is for a number))
make -jN tucana_user_defconfig
make menuconfig (configure config to your liking)
use save button (highlight save and press enter, can use arrow key right and left as well)
make -jN
Or it could be
make -jN ARCH=arm64
After doing all this, for me the build fails. Its driving me insane!
I think I also was using the wrong AOSP kernel, was using common-4.14 and now using this one
refs/heads/q-common-android-4.14 - kernel/manifest - Git at Google
guessing the Q is for Android 10
MY DEMON!!!!!
1615228944041.png
i think it is not working because i am using Android 10 R41 and should be using Android 10 R29. Testing it now.
Alright, went from ubuntu version 20 to 18 (Bionic is more reliable and easier to set up).
Had 20, due to bionic auto updated (my own fault).
Version 20 does work with some modding and adding bionic to the repo's of ubuntu.
I tried using AOSP toolchain and also Qualcomm's (QQ-LLV) LLV toolchain 8.0 for clang.
The part from MiCode wiki on github is saying to use CLANG_Triple with aarch64-linux-gnu which I can not find this file anywhere.
Not in QQ-LLV, AOSP-10-R29 aarch64 that is located in prebuilts > gcc
so the main problem I am getting is the cpu timer failing, during the build process. I try to modify the config by typing "make menuconfig" and then change the cpu govern type from performance to ondemand and saving it as .config (not sure if it is supposed to be the same name as the tucana-user-defconfig, let me know if this is the problem)
the AOSP android version I am using is R29. The AOSP kernel I am using is common-4.14
none of these files have aarch64-linux-gnu.
I am starting to slowly give up on this whole thing. spent 7 days just to get a cpu timer problem during build.
Oh and the source i am using is from Micode github under tucana for android 10 (Q)
Just sprung up an idea. I think i am supposed to first use Qualcomms LLVM toolchain 8.0 with the Mi source code package to make up the files needed in order to use anything from AOSP. please let me know if this is correct.
No matter what guide i find for it, it shows CLANG_TRIPLE=aarch64-linux-gnu- and every time. it just can not find it. no idea how to get this.
it work in clang triple for -gnu
AOSP-Q-Kernel-4.14.117
refs/heads/q-common-android-4.14 - kernel/manifest - Git at Google
Spoiler: Kbuild config
ARCH=arm64
SUBARCH=arm64
BRANCH=android-4.14
CLANG_TRIPLE=aarch64-linux-gnu-
CROSS_COMPILE=aarch64-linux-android-
DEFCONFIG=tucana_user_defconfig
KERNEL_DIR=xsource
DTC_EXT=dtc
DTS_EXT=dts
CC=clang
LZ4_RAMDISK=1
POST_DEFCONFIG_CMDS=""
EXTRA_CMDS=''
CLANG_PREBUILT_BIN=prebuilts-master/clang/host/linux-x86/clang-r353983c/bin
LINUX_GCC_CROSS_COMPILE_PREBUILTS_BIN=prebuilts/gcc/linux-x86/aarch64/aarch64-linux-android-4.9/bin
REAL_CC=prebuilts/ndk/toolchains/llvm-Snapdragon_LLVM_for_Android_8.0/prebuilt/linux-x86_64/bin/clang
BUILDTOOLS_PREBUILT_BIN=build/build-tools/path/linux-x86
FILES="
O=/out
arch/arm64/boot/Image.gz
vmlinux
System.map
"
STOP_SHIP_TRACEPRINTK=1
OMFG!!!!!!! VICTORY!!!!!!!!
I think.
1615380568663.png
MAYBE!!!!!
1615380671546.png
Spoiler: New info
Alright Managed to fix most of the problems in the OLD area, mostly due to path issues, always check your paths (PATH).
Spoiler: build.config
ARCH=arm64
SUBARCH=arm64
BRANCH=K4.14Q
CLANG_TRIPLE=aarch64-linux-gnu-
CROSS_COMPILE=~/android/xkernel/tsource/toolchains/gcc/linux-x86/aarch64/aarch64-linux-android-4.9/bin/aarch64-linux-android-
CROSS_COMPILE_ARM32=~/android/xkernel/tsource/toolchains/gcc/linux-x86/arm/arm-linux-androideabi-4.9/bin/arm-linux-androideabi-
KBUILD_DEFCONFIG=~/android/xkernel/tsource/arch/arm64/configs/tucana_user_defconfig
DEFCONFIG=tucana_user_defconfig
POST_DEFCONFIG_CMDS="check_defconfig"
DTC_EXT=dtc
DTC_PREBUILTS_BIN=/scripts/dtc
KBUILD_OUTPUT=out
HOSTCC=gcc
CC=clang
AS=clang
AR=ar
CLANG_PREBUILT_BIN=/toolchains/clang/host/linux-x86/clang-r353983c/bin
BUILDTOOLS_PREBUILT_BIN=/toolchains/build-tools/linux-x86/bin
FILES="
arch/arm64/boot/Image.gz
vmlinux
System.map
"
Don't know if most of what is in there is needed or if everything I need is there but that is what i have so far.
One thing that i know always does not work is the check_defconfig. fails to match every time.
Tried using ld.ldd and it just keeps saying the vmlinux file size is too large. I have not given up and can say i have learned a lot.
Will update more as I go.
Spoiler: Links
Toolchain and kbuild config help
Hello I have been trying to extract the kernel from Tucana android 10 source. I would like to know if anyone has a working config to be able to build up the kernel. I have a config but it does not extract everything, well I don't think it does...
forum.xda-developers.com
[SOLVED] dts not found
Hello. Been working on how to get a device kernel from source for the Mi note 10 pro (Mi CC9 Pro). I have gotten up to the point where it builds but fails due to dts folder is not found. I need an example of what goes in DTC_EXT= All I see on...
forum.xda-developers.com
Spoiler: UPDATE 2022 WORKING BUILD WITH ISSUES
Alright I decided to give it another shot in 2022 because of getting replies.
1. Get your source from device brand. (https://github.com/MiCode/Xiaomi_Kernel_OpenSource)
2. Use Mi code wiki to learn how to build (https://github.com/MiCode/Xiaomi_Kernel_OpenSource/wiki)
3. Go right hand side for standalone kernel. Because the how to section just waste hours on end with nothing built and Soong and clang errors.
4. Follow msm-4.14 for a guide. (https://github.com/MiCode/Xiaomi_Kernel_OpenSource/wiki/How-to-compile-kernel-standalone)
5. Make sure to have dtc binary file from aosp source (https://android.googlesource.com/platform/prebuilts/misc/) choose your android version. Check device for android version. Can use CPU ID in playstore under System, section API LEVEL.
6. Get llvm Snapdragon from Qualcomm. (https://developer.qualcomm.com/software/snapdragon-llvm-compiler-android) I used both but you should be able to use 8.0. both have off same results during the build process.
7. Get Aosp gcc
(https://android.googlesource.com/platform/prebuilts/gcc/linux-x86/aarch64/aarch64-linux-android-4.9/)
8. Put LLVM Snapdragon toolchain and AOSP gcc into a folder in root of kernel folder called toolchain.
9. Run the script mi provided. Change the 6.0 on the last 2 make commands to 8.0 unless you are already using 6.0, then you can leave it at 6.0.
10. Build.
11. Get a boot.img extractor from github or on these forums. I can't recommend one yet, due to not having full success. Can search boot.img extractor or boot.img unpacked as seperate search terms.
**Recommendation**
Boot and Recovery
GitHub - xiaolu/mkbootimg_tools: Unpack and repack boot.img,support dtb(dt.img).
Unpack and repack boot.img,support dtb(dt.img). Contribute to xiaolu/mkbootimg_tools development by creating an account on GitHub.
github.com
OR
Android Kitchen
GitHub - osm0sis/Android-Image-Kitchen: Automated scripts to unpack/repack Android kernel/recovery images + ramdisks
Automated scripts to unpack/repack Android kernel/recovery images + ramdisks - GitHub - osm0sis/Android-Image-Kitchen: Automated scripts to unpack/repack Android kernel/recovery images + ramdisks
github.com
OR
Android Kitchen
GitHub - cfig/Android_boot_image_editor: Parsing and re-packing Android boot.img/vbmeta.img/payload.bin, supporting Android 13
Parsing and re-packing Android boot.img/vbmeta.img/payload.bin, supporting Android 13 - GitHub - cfig/Android_boot_image_editor: Parsing and re-packing Android boot.img/vbmeta.img/payload.bin, supp...
github.com
12. Extract boot.img from stock rom.
13. Put image.gz.dtb in unpacked boot image. Delete original file and rename new one as the same name.
14. Repack boot.img and either upload using fastboot or use twrp to install the boot.img
The problem I have at the moment is I am unable to use touch input at all. It boots loads system but can't touch anything. No input at all. Hardware buttons work though.
Might be solution on 3rd post to no touch input. https://forum.xda-developers.com/t/reference-how-to-compile-an-android-kernel.3627297/page-43
VMLINUX FIX: HERE
Hi @Squida, I've also been on this road, I just hit a milestone when I actually have something booting (but not much working), see the thread I started if it works for you too
b100dian said:
Hi @Squida, I've also been on this road, I just hit a milestone when I actually have something booting (but not much working), see the thread I started if it works for you too
Click to expand...
Click to collapse
Spoiler: Reply
Hey mate, thanks for the reply. Could you put a link to your thread so I can check it out.
Found it: https://forum.xda-developers.com/t/building-lineageos-17-1-from-source.4245417/
I also have posted on Qualcomm forums.
LINK: https://developer.qualcomm.com/forum/qdn-forums/software/snapdragon-llvm-compiler-android/68403
I found out qcomm has its own builder for msm devices. It's known as QAEP. I have been trying to build the sm6150 with Tucana defconfig. The thread above is the issue I have in trying to build it. No idea how to get "SDClang" so as soon as I work that out. It should build with QAEP instead of using AOSP.
There's a commit where I add SDCLANG support (but I don't think that's needed, is backed out atm) https://github.com/alibei/android_d...mmit/efbb3c66aa0d385e58052675402489c13392576e )There's a similar commit in the kernel.
You need to register to Qualcomm to download it, and unzip it in kernel/toolchains, just as https://github.com/MiCode/Xiaomi_Kernel_OpenSource/wiki/How-to-compile-kernel-standalone says under "You must get llvm clang from qcom". You just have to get the 8.x version.
Some snippets for the commands I've tried https://gist.github.com/b100dian/40c8dbe746ff181aff71ee10a75a5f3c
Spoiler: Reply
Yeah I am not trying to use Lineage sources. so far steps I have taken are as follows.
1. Mi source code download of Tucana android 10, data source, wifi source, audio source.
2. Extracted Mi tucana source in a directory (also tried the git way to update CAF tags).
3. using AOSP as the source for build. I tried using Clang with cross compile with gcc (GNU)
4. with using all AOSP toolchains so clang and gcc from AOSP and then using LLVM clang as reall cc, due to it does not contain clang itself.
all that for kernel and it builds but I feel its missing drivers. due to the warnings that Qcom gives. also got wifi modules installed but not audio. Audio source is a little different then the wifi source.
For the proprietary binaries. I used lineage Extract script with the lineage 17.1 tucana proprietary-files.txt list.
extraction worked on miui_TUCANAGlobal_V12.0.4.0.QFDMIXM_be49be8fa0_10.0.zip.
but of course the device tree is missing, found the platform sm6150 device tree on QAEP. so now trying to use QAEP to build not only the kernel but the rom as well using QAEP instead of AOSP. think we need to use QAEP, then using the files built. can then move over to aosp for upgrading, etc.
Could you explain this error considering you managed to get SDClang working. it may solve my problem. error provided below.
Spoiler: sdclang error
I have a SDCLANG_PATH set in BoardConfig.mk in the commit I pointed to earlier.
Basically is from a toolchains folder creaded with `tar -xvzf snapdragon-llvm-8.0.6-linux64.tar.gz` in the kernel/xiaomi/tucana folder
b100dian said:
I have a SDCLANG_PATH set in BoardConfig.mk in the commit I pointed to earlier.
Basically is from a toolchains folder creaded with `tar -xvzf snapdragon-llvm-8.0.6-linux64.tar.gz` in the kernel/xiaomi/tucana folder
Click to expand...
Click to collapse
what i have setup in boardconfig is as follows.
ifneq ($(HOST_OS),linux)
SDCLANG := true
SDCLANG_PATH := toolchain/ndk/android-ndk-r22/toolchains/llvm-Snapdragon_LLVM_for_Android_8.0/prebuilt/linux-x86_64/bin
SDCLANG_LTO_DEFS := device/qcom/common/sdllvm-lto-defs.mk
endif
this has been added just above wifi. still fails with same error.
Spoiler: Boardconfig.mk
Image:
I think I just solved it, so i noticed you talk about Boardconfig.mk and when i checked out your git code, it had in the same config the target. well QAEP is a little different it not only has a Boardconfig file but also a AndroidBoard config file. so i think i am supposed to be adding it in there instead of Boardconfig. testing it now.
Spoiler: AndroidBoard.mk
Something else i also noticed that is not working.
Spoiler: .ko files missing
And can confirm above configuration changes are not working, not sure where it is located to tell it the path.
I cannot speak for QAEP (I barely began reading about lineage but in my case I don't have out/target/product/sm6150 at all, only out/target/product/tucana, which seems to be the name of the kernel (or device or vendor). Do you also have other repos pulled in that contain kernel named sm6150? Like https://github.com/LineageOS/android_kernel_xiaomi_sm6150 Maybe you should not have both
b100dian said:
I cannot speak for QAEP (I barely began reading about lineage but in my case I don't have out/target/product/sm6150 at all, only out/target/product/tucana, which seems to be the name of the kernel (or device or vendor). Do you also have other repos pulled in that contain kernel named sm6150? Like https://github.com/LineageOS/android_kernel_xiaomi_sm6150 Maybe you should not have both
Click to expand...
Click to collapse
Spoiler: b100dian reply
So where yours is out/target/product/tucana, the one for QAEP is SM6150 which is in the same location. so I think maybe on the right track here. also thanks for the help for llvm dragon, had to set bin path, for it to work. but i have not gone a built the rom yet, doing the kernel build first now.
Instead of relying on QAEP builder. I went and got kernel/build from code-aurora for android 10 r40 and ended up getting misc linux folder in kernel also gcc and build tools from the device tag release repo on codeaurora.
Usually i was using AOSP but now switched over to all QAEP tools and sources.
I am testing the Micode Audio and wifi source. got wifi working during kernel build by adding this below.
EXT_MODULES="
mods/wlan/qcacld-3.0
"
IN_KERNEL_MODULES=1
I believe that installs the wifi drivers for the device as a module. but building with clang, gcc and gnu as clang tripple this is all related to only kernel building.
Spoiler: Update
Done an overhaul on everything for the kernel. decided to switch from QAEP to AOSP.
Reason for the switch, I noticed with AOSP I am able to not only download the kernel/msm4.14 common folder but it also downloads build, prebuilts, prebuilts-master folders with everything included.
Differences aside from what I mentioned above.
AOSP common-4.14 comes with up to date builder, gcc, clang, etc. But when I download the release tag for example, LA.UM.9.1.r1-06700-SMxxx0.0 at codeaurora on the otherhand, it only downloads the common folder and no extra folders for it.
So then in turn. you have to go on codeaurora and download by using git clone, the build folder for what ever android version it is. for example, I had android version 29 (r29) though it does not exist in the branch list on codeaurora so i went with 28.
I should also note that making the change from QAEP to AOSP I started again with the kernel source.
With making the change, I have noticed improvements right away, but also I noticed that having CC=clang, the build would not work untill I also put in HOSTCC=gcc. guessing its to do with the AOSP version being a little different. still using QualComms LLVM clang compiler for everything clang related.
Spoiler: Problems I have so far.
***Error-1***
Have it building but get this spam, been trying to solve it.
***Error-2***
This error pops up when using LD=ld.lld
this is what is set in the build.config
***CONFIG-1***
LD=ld.lld
LD_LIBRARY_PATH=toolchain/llvm-Snapdragon_LLVM_for_Android_8.0/prebuilt/linux-x86_64/lib/clang/8.0.6/lib/linux/aarch64:$LD_LIBRARY_PATH
export O=out/android-4.14 LD_LIBRARY_PATH
**RESULTS**
***CONFIG-2***
LD=ld.lld
HOSTLDFLAGS="-fuse-ld=lld" (added it so i can modify)
LD_LIBRARY_PATH=toolchain/llvm-Snapdragon_LLVM_for_Android_8.0/prebuilt/linux-x86_64/lib/clang/8.0.6/lib:$LD_LIBRARY_PATH
export O=out/android-4.14 LD_LIBRARY_PATH
**RESULTS**
Same as above.
Spoiler: Config commands
This right here may have just solved all my problems. can confirm in all my config tries. Never once thought to put aarch64-linux-android before all variable values.
SOURCE: https://developer.android.com/ndk/guides/standalone_toolchain
# Tell configure what tools to use.
target_host=aarch64-linux-android
export AR=$target_host-ar
export AS=$target_host-clang
export CC=$target_host-clang
export CXX=$target_host-clang++
export LD=$target_host-ld
export STRIP=$target_host-strip
Also discovered that LD=ld.lld is for clang compiler. could be why the above Problem is happening.
This is using the toolschains without NDK, may have to put qualcomm and gcc in NDK and path them.
So after trying to downgrade to android 10 from 11, had no luck in doing so.
So now sticking with android 11 and I can safely say that I have reached a mile stone.
I ended up downloading android 11 R30 and using all the tool chains from AOSP while still using Qualcomm's LLVM compiler 8.0.6 as REAL_CC. But the clang pre-built path is clang from AOSP. Just trying to work out how to include the Xiaomi audio source. Unable to work out where to put it. Still testing, have not given up.
Hi @Squida, where is `mods/wlan/qcacld-3.0` from, the qcom talos referenced above in BoardConfig.mk?
Do you have an exact link to the repo?
In the meantime I can confirm I am missing something form the kernel. If I build everything myself and replace _just_ the kernel (with my dtb appended etc) it has sound/wireless, so knowing what wifi / audio module to link in seems the way to go.
b100dian said:
Hi @Squida, where is `mods/wlan/qcacld-3.0` from, the qcom talos referenced above in BoardConfig.mk?
Do you have an exact link to the repo?
In the meantime I can confirm I am missing something form the kernel. If I build everything myself and replace _just_ the kernel (with my dtb appended etc) it has sound/wireless, so knowing what wifi / audio module to link in seems the way to go.
Click to expand...
Click to collapse
Spoiler: Reply
No problem at all, anything to get this build done lol.
All links are for Android 11
If you want Android 10, replace pheonix-r-oss with tucana-q-oss
**Example for Android 10**
GitHub - MiCode/Xiaomi_Kernel_OpenSource at tucana-q-oss
Xiaomi Mobile Phone Kernel OpenSource. Contribute to MiCode/Xiaomi_Kernel_OpenSource development by creating an account on GitHub.
github.com
******ANDROID 11******
***SOURCE***
GitHub - MiCode/Xiaomi_Kernel_OpenSource at phoenix-r-oss
Xiaomi Mobile Phone Kernel OpenSource. Contribute to MiCode/Xiaomi_Kernel_OpenSource development by creating an account on GitHub.
github.com
***DATA***
GitHub - MiCode/vendor_qcom_opensource_data-kernel at phoenix-r-oss
xiaomi opensource for data-kernel. Contribute to MiCode/vendor_qcom_opensource_data-kernel development by creating an account on GitHub.
github.com
***AUDIO***
GitHub - MiCode/vendor_qcom_opensource_audio-kernel at phoenix-r-oss
Contribute to MiCode/vendor_qcom_opensource_audio-kernel development by creating an account on GitHub.
github.com
***WIFI (mods/wlan/qcacld-3.0)***
MiCode/vendor_qcom_opensource_wlan
Contribute to MiCode/vendor_qcom_opensource_wlan development by creating an account on GitHub.
github.com
Thanks, I think I get now what you're saying with 'talos', I poked at those repos now.
It seems to me the modules are built out of tree, as they don't appear in the extracted kernel config. (See https://github.com/b100dian/Xiaomi_Kernel_OpenSource/commit/0f4b062ef806aaad6ad02e2efd87809e8b7250c6).
Also, having dlkm from https://source.codeaurora.org/quic/la/device/qcom/common/tree/?h=LA.UM.8.9.r1-03800-sm6150.0 does not seem to help automate the build, I get all sorts of errors when AndroidKernelModule.mk is included and it includes back the AndroidKernel.mk..
Getting these into drivers/staging seems easier at first but..for now, I've only managed to build wlan.ko but that still errors out with `wlan: disagrees about version of symbol module_layout` :-S
b100dian said:
Spoiler: Reply
Thanks, I think I get now what you're saying with 'talos', I poked at those repos now.
It seems to me the modules are built out of tree, as they don't appear in the extracted kernel config. (See https://github.com/b100dian/Xiaomi_Kernel_OpenSource/commit/0f4b062ef806aaad6ad02e2efd87809e8b7250c6).
Also, having dlkm from https://source.codeaurora.org/quic/la/device/qcom/common/tree/?h=LA.UM.8.9.r1-03800-sm6150.0 does not seem to help automate the build, I get all sorts of errors when AndroidKernelModule.mk is included and it includes back the AndroidKernel.mk..
Getting these into drivers/staging seems easier at first but..for now, I've only managed to build wlan.ko but that still errors out with `wlan: disagrees about version of symbol module_layout` :-S
Click to expand...
Click to collapse
Spoiler: Reply
It seems you are having a similar issue.
When doing everything I have done so far, The biggest wall for me is getting <LD=LD.lld> to work properly, I either get clang error with a file having a linking problem with using clang from QQ llvm compiler. And when I switch over to using AOSP's 4.14 stable kernel prebuilts-master clang. It works but then another error to do with CPU timer pops up. Which was originally because of using gcc instead of clang in the CC=clang environment variable, or removing it out of the build.config file.
i myself did a major overhaul of Ubuntu and have upgraded to LTS 20.04 from 18 bionic.
Noticed some things right away.
When you install gcc-multilib and g++-multilib. You get version 9 instead of you being on Ubuntu bionic and getting version 7. Also faster in general with general use of the operating system using Oracle vmbox. On windows 10.
Apart from linking errors, kernel is buding but only with not having LD=LD.lld in the build.config file.
Spoiler: wifi-info
And for the wifi qcom. Mods folder I added manually. Inside the devices source folder. Then copied WLAN sources inside and renamed the folder to just WLAN.
Spoiler: where I am at now
I have got both android 11 and android 10 kernel and ROMs. Also went and downloaded the kernel for coral. That's how I found out about the wifi and how I got a lot of the settings for the build.config.
I am gonna have a break on this whole thing, gotten to a point where with android 11 Xiaomi Phoenix. I get one error with a file in kbuild, using the same config settings for android 10. I get none. Apart from the qcom space which is either DTC not working or failing at some point. Not sure if we need to use LD=LD.lld I have been reading it is not needed due to the builder choosing the correct one for you.
Overall though it builds. Just the spam for qcom warnings I just can not get rid of and it's to do with the sensors.
If anyone has information on what causes the qcom spam. Please let me know, thank you.
Plus this all started with vmlinux not working right. Something to do with channel scratch. Can not fix it.
Spoiler: My Thoughts
I have just found this exploring info on the rom.
Build AOSP with LineageOS device tree
My device (Xiaomi Redmi Note 5, whyred) have official LineageOS support and therefore there is device tree and kernel. I want to build AOSP without any modifications or tweaks. How can I use (or po...
stackoverflow.com
Key part taken from above link.
**Its hard to build pure AOSP than other ROMs. While building a custom ROM a lot of components wont work**
I think for our devices we have no choice but to use QAEP instead of AOSP, it explains why it fails to build properly. we first need a full working rom and then I think we can move over to AOSP and modify lets say, coral device and match it for our device. Though I think it is just easier using QAEP and why QAEP exists in the first place.
When I used QAEP, I got the SDCLANG problem, the build process according to the Micode git wiki, section "How to use" explains on how to configure and by the looks of the guide. It seems simple as cooking a boiled egg with only a couple of modifications to a device tree and adding your kernel to the kernel folder. It is supposed to build. I myself get SDClang path problems when using QAEP though.
I will go back to QAEP but i will be using all android 11 sources. due to unable to downgrade my phone from android 11 to 10. Having issues at the moment with TWRP installing. Shows my folder structure folder and file names. all random characters.
I will update as i get further in building the kernel and rom
Spoiler: Thoughts2
So I am now merging the built kernel into the kernel directory of AOSP 10. Made a folder called common and moved the built kernel into it. Also made a folder called prebuilts and put in all the .gz into it under a folder called 4.14. discovered that the AOSP 10 picked up the common folder with the built kernel but is having errors with Android.bp, will update as I find out more. Right now, just tinkering.
This is the issue when you have the build tools outside the source folder.
Could be a path I have not set properly. once I removed the command from POST_DEFCONFIG_CMDS="" it is now building.
Couple of things to note.
REAL_CC= I believe has now changed to HOSTCC=. Command is not found in any files in the build directory of the AOSP 4.14 STABLE Kernel.
QualComms LLVM 8.0.6 Compiler does not contain clang files. you still need to either download it, or just use the one in AOSP 4.14 Stable kernel, plus the clang version in AOSP stable kernel is a later version and also contains clang version for android 10+ instead of android 9.
New error I get after changes mentioned above are made.
Not sure where to put -fPIC to make the command function.
Fixed it, was a pathing issue with folders.
But now I am stuck now on finding out the directory for LD_LIBRARY_PATH=
Code:
DTC arch/arm64/boot/dts/qcom/apq8016-sbc.dtb
dtc: error while loading shared libraries: /home/avm/dev/source/kernel/prebuilts-master/ndk/toolchains/llvm-Snapdragon_LLVM_for_Android_8.0/prebuilt/linux-x86_64/lib/libc++.so: file too short
make[4]: *** [scripts/Makefile.lib:325: arch/arm64/boot/dts/qcom/apq8016-sbc.dtb] Error 127
make[3]: *** [/home/avm/dev/source/kernel/mi10/scripts/Makefile.build:676: arch/arm64/boot/dts/qcom] Error 2
make[2]: *** [arch/arm64/Makefile:187: dtbs] Error 2
make[2]: *** Waiting for unfinished jobs....
CC lib/rhashtable.o
Spoiler: Extra Discovered Info
${ROOT_DIR} is used. With it and you don't need to export it. Builder already knows what it is. so you use it like this for example
************************************
CC=${ROOT_DIR}/path-to-QQ_LLVM_COMPILER clang binary file, located in bin folder.
Also do this for ld.lld
LD=${ROOT_DIR}/path-to-ld.lld
************************************
Found another odd thing that happens
Having CC=PATH-TO-QQ-llvm-compiler/clang with CC_PREBUILT_BIN as aosp's location and HOSTCC=clang
It builds.
But with LD=
It fails.
I try using QQ-LLVM-Compiler as CC_Prebuilt_bin=
Fails.
So far I have managed to get to this point.
And can safely say that using QualComs LLVM 8.0.6 compiler does not work with android 10 so stick with AOSP's prebuilt clang either in Kernel stable git or AOSP android-Number-Revision
Again, REAL_CC does not work, nor show up in the enviroment as a used command.
These below all do.
<CC=>, <HOSTCC=>, <HOSTLDFLAGS=-fuse-ld=lld>, <NM=llvm-nm>, <OBJCOPY=llvm-objcopy>, <HOSTLD=>, <LD=>,
HOSTCC= will use what is put in CLANG_PREBUILT_BIN=
CC= you are able to add QQ-LLVM-COMPILER's clang and it will run.
Proof of QQ-LLVM-COMPILER failing, when using it for <CLANG_PREBUILT_BIN=>
Instead of using AOSP's clang.
So far that is all I have seen pop up.
All these changes were done with the Kernel AOSP-COMMON-4.14-STABLE build, kernel, prebuilts, prebuilts-master folders and then with the devices source in a folder with a build.config file in the same location as the AOSP-kernel build folders. this all seems to work, Only thing I lose out on, is using POST_DEFCONFIG_CMDS="" to use make menuconfig.
Reason that make does not work is because according to AOSP coral's kernel you have another build.config file which is with the build folder, prebuilts, master-prebuilts folders which connects to the main build.config.common and clang config that has the extra commands. So in turn I do not think we use LD=ld.lld
We let the builder choose it for us.
95% sure that the DTC is what is causing qcom to have that warning spam.
Device tree compiler (DTC) linking seems to be the full cause of all the issues.
Okay to prevent fixdep.c:105:10: error all you need to do is have HOSTCC=gcc and it works. so then in turn. You can now use QQ LLVM Clang compiler in CLANG_PREBUILT_BIN= instead of AOSP.
Spoiler: Helpful Links
GitHub - nathanchance/android-kernel-clang: Information on compiling Android kernels with Clang
Information on compiling Android kernels with Clang - GitHub - nathanchance/android-kernel-clang: Information on compiling Android kernels with Clang
github.com
Spoiler: Still going
Still going at it, manage to find out a couple of things.
If your build is failing due to files missing, check your build directory and make sure the files in the path folder and in Linux-x86>bin folders are indeed "symlinked" something I was unaware of with Linux due to me being a windows user. none of the icons can have an X on it. it means its broken. this solved a ton of my issues during Rom building.
the build folder itself is a symlink folder. (Not all files)
Just locate and put files in correct places. so instead of having a folder called toolchains, like we are told on the Micode wiki, etc. If you use the AOSP-kernel build, prebuilts, prebuilts-master. which contains all kernel and build tools. and are already linked. so then you just add QQ-LLVM-Compiler to either prebuilts or pre-builts master and link it in the config.
If you do all this correctly and having all files in the proper location, you should not have any more build errors in regards to missing files.
Summed up
Build directory contains symlinks to binary files which are located in prebuilts and prebuilts-master. you need all 3 folders in the device source root directory.
Inside the build directory you will find a file called build.config, its not a config. its a symlink for one. Rename it to the build.config.Aarch64 for example, or what ever your config is called located in your root devices source directory. this solved a **** load of my problems.
The problem I have now is actually got to do with Repo, AOSP, QAEP. found out using the manual install method of repo makes it so you can download off google when you have the pgp key installed. but having it installed this way, stuffs up gpg for QAEP. I am looking for a way to merge all PGP keys in one location.
and to get QAEP downloading without showing this error.
All you have to do is remove the manual repo you installed and then also deleting the .repoconfig folder and .gnupg folder.
Then install repo with <sudo snap install git-repo>
it will work, you will get a public key under john doe and it will use CAF as well. problem is. You lose out on public key to AOSP and also the repo that is inside the AOSP folder needs to be replaced. trying it just causes errors though.
What happens when you do above by removing the manual repo and adding the snap git-repo
So why does it matter that I need both public keys to work?
Because I am using AOSP and also QAEP for testing.
I'll be sticking with the snap instal git-repo for now. i'll have to work out how to add the key from AOSP.
**Error when going back to AOSP**
**what happens when you delete the repo folder inside .repo of AOSP**
Only way I got it to work is by removing the snap install version of repo and install the manual way.
But then I lose out on CAF.
Think i may have worked out what is happening with the keys. 2 folders are being made. .repoconfig and .gnupg.
.gnupg only gets created when using AOSP pgp key.
and there is a gnupg folder in .repoconfig, might be a command to merge the files.
this command below makes it so you can change the name.
--config-name
put it at the end of the repo init command.
from what I understand for the tucana_user_defconfig.
#CONFIG_BUILD_ARM64_DT_OVERLAY does not need to be set. for mi 9, yes. not for mi note 10. Just pulled the defconfig from my device using the command below.
**EXTRACT DEFCONFIG FROM DEVICE**
MSM devices
CMD> adb pull /proc/config.gz
files appears as config.gz in same directory
I should also note that I ran 2 builds one with DT_OVERLAY=Y and then one =N
the outcome was with the change of CONFIG_BUILD_ARM64_DT_OVERLAY=n
That is not how it is setup, the proper way to disable it is as follows.
#CONFIG_BUILD_ARM64_DT_OVERLAY is not set
I got a bigger file size for Image.gz-dtb. instead of under 20MB
Have it on so you can extract DTBO and dtb files. having it off stops it from extracting.
THIS error or what ever the hell it is, is driving me insane.
My understanding is that DTBO (overlay), when enabled, ends up in dtbo.img, and when disabled, probably ends up in dtb which is appened to the kernel Image.gz.
The gsi_write_channel_scratch error is... maybe this helps? https://github.com/ClangBuiltLinux/linux/issues/931#issuecomment-599681910
Spoiler: reply
Btw, thank you @Squida for pointing me out the correct repos for wifi and audio, I managed to compile both into my build and the audio one inline (the wifi needs to be insmod'ed).
b100dian said:
My understanding is that DTBO (overlay), when enabled, ends up in dtbo.img, and when disabled, probably ends up in dtb which is appened to the kernel Image.gz.
The gsi_write_channel_scratch error is... maybe this helps? https://github.com/ClangBuiltLinux/linux/issues/931#issuecomment-599681910
Spoiler: reply
Btw, thank you @Squida for pointing me out the correct repos for wifi and audio, I managed to compile both into my build and the audio one inline (the wifi needs to be insmod'ed).
Click to expand...
Click to collapse
Spoiler: Reply
You sir, are a Legend!
Okay So if I am reading that right.
If you mean DT_OVERLAY=y in defconfig, makes it so I can get the dtbo.img file. I tried adding the dtbo.img to the list of files for extraction and with it on, no dtbo image is made nor found, with it off. still the same thing but unable to extract any dtb or dtbo images from the boot directory. have to have it on.
dtbo.img just does not want to extract for me.
Again, using the clang config and not the mi 9 config used on Micode wiki.
the Micode Wiki settings do not work correctly. you lose out on about a GB of stuff and I know its a DTC issue I have been having due to the Qcom spam.
I have reduced the space though so now you can actually scroll through the whole build process, including the spam.
I will definitely be looking into that solution for channel scratch.
Also if you could please show me the config for the SDClang settings you put it to get it to detect. I tried looking over your git page and failed to understand why you have it in boardconfig.mk and if lineage has other files for the device.
Reason being is because on QAEP it talks about SDclang-3.8 and i have no idea where it is.. Supposed to be in QQ-LLVM-compiler and you copy it to the prebuilts folders, it just does not exist to do that.
the major problem I have is building the device tree when building the kernel. from my understanding, Xiaomi have set it up so DTC creates a Device tree for you. well that is the part that is failing and the whole reason why I can not make a rom.
My guess because you are using lineage sources, everything I am talking about. they already did for you. I am trying to do it all manually. learning purposes.
Okay so it turns out, we need LD=ld.lld
Though we must modify files, this is ridiculous just to get your own rom for xiaomi devices.
This thread below may have the solution for me and an easier one then having to modify files manually. Might have to update the Devices Kernel to the stable aosp 4.14 kernel. this is my theory anyway.
[REFERENCE] How to get an Android kernel up to date with linux-stable
Introduction Hello everyone! This will be a thread to assist people with getting their device's Android kernel up to date with the latest linux-stable tag from kernel.org. This process will henceforth be referred to as "upstreaming". This thread...
forum.xda-developers.com
Squida said:
Spoiler: Reply
You sir, are a Legend!
Okay So if I am reading that right.
If you mean DT_OVERLAY=y in defconfig, makes it so I can get the dtbo.img file. I tried adding the dtbo.img to the list of files for extraction and with it on, no dtbo image is made nor found, with it off. still the same thing but unable to extract any dtb or dtbo images from the boot directory. have to have it on.
dtbo.img just does not want to extract for me.
Again, using the clang config and not the mi 9 config used on Micode wiki.
the Micode Wiki settings do not work correctly. you lose out on about a GB of stuff and I know its a DTC issue I have been having due to the Qcom spam.
I have reduced the space though so now you can actually scroll through the whole build process, including the spam.
I will definitely be looking into that solution for channel scratch.
Also if you could please show me the config for the SDClang settings you put it to get it to detect. I tried looking over your git page and failed to understand why you have it in boardconfig.mk and if lineage has other files for the device.
Reason being is because on QAEP it talks about SDclang-3.8 and i have no idea where it is.. Supposed to be in QQ-LLVM-compiler and you copy it to the prebuilts folders, it just does not exist to do that.
the major problem I have is building the device tree when building the kernel. from my understanding, Xiaomi have set it up so DTC creates a Device tree for you. well that is the part that is failing and the whole reason why I can not make a rom.
My guess because you are using lineage sources, everything I am talking about. they already did for you. I am trying to do it all manually. learning purposes.
Click to expand...
Click to collapse
Spoiler: Reply
My guess because you are using lineage sources, everything I am talking about. they already did for you. I am trying to do it all manually. learning purposes.
Click to expand...
Click to collapse
Of course, this is the reason. I don't know exactly how the lineage build scripts generate dtbo.img. There's make bootimage, make vendorimage and maybe make dtboimage too. Probably this command helps you generate it, but I don't think I had success with that: https://forum.xda-developers.com/t/...nel-dtbo-for-redmi-k20.3973787/#post-80354635
The image size differs and the one created by lineage build scripts is the same size as the original one.
I also have console spam when the device tree is generated, I just didnt sweat on it as being an error.
BoardConfig is probably central to the lineage build, but I assume the variables set here are available to kernel's make commands.
For a lineage-less built I used the first two commands here: https://gist.github.com/b100dian/40c8dbe746ff181aff71ee10a75a5f3c (the rest of the things are my attempts to construct back the boot.img with the kernel).
To actually boot the kernel you can gzip it, and append the dtb file to it (
like
Code:
cat Image.gz dtb > Image.gz-dtb
).
, and _then_ reconstruct a deconstructed original boot.img with that (w/o the --dtb parameter if I remember correctly). But I think kernel output already has -dtb concatenated in out/arch/arm64/boot
b100dian said:
Spoiler: Reply
Of course, this is the reason. I don't know exactly how the lineage build scripts generate dtbo.img. There's make bootimage, make vendorimage and maybe make dtboimage too. Probably this command helps you generate it, but I don't think I had success with that: https://forum.xda-developers.com/t/...nel-dtbo-for-redmi-k20.3973787/#post-80354635
The image size differs and the one created by lineage build scripts is the same size as the original one.
I also have console spam when the device tree is generated, I just didnt sweat on it as being an error.
BoardConfig is probably central to the lineage build, but I assume the variables set here are available to kernel's make commands.
For a lineage-less built I used the first two commands here: https://gist.github.com/b100dian/40c8dbe746ff181aff71ee10a75a5f3c (the rest of the things are my attempts to construct back the boot.img with the kernel).
To actually boot the kernel you can gzip it, and append the dtb file to it (
like
Code:
cat Image.gz dtb > Image.gz-dtb
).
, and _then_ reconstruct a deconstructed original boot.img with that (w/o the --dtb parameter if I remember correctly). But I think kernel output already has -dtb concatenated in out/arch/arm64/boot
Click to expand...
Click to collapse
Thanks for the quick reply.
Spoiler: Might be the solution to DTC
Code:
DTC_EXT=~/android/lineage/prebuilts/tools-lineage/linux-x86/dtc/dtc ARCH=arm64 SUBARCH=arm64 CROSS_COMPILE=${PWD}/toolchain/bin/aarch64-linux-android- make O=../tucana-out REAL_CC=${PWD}/toolchains/llvm-Snapdragon_LLVM_for_Android_8.0/prebuilt/linux-x86_64/bin/clang CLANG_TRIPLE=aarch64-linux-gnu - vendor/tucana_user_defconfig
DTC_EXT=~/android/lineage/prebuilts/tools-lineage/linux-x86/dtc/dtc ARCH=arm64 SUBARCH=arm64 CROSS_COMPILE=${PWD}/toolchain/bin/aarch64-linux-android- make -j8 O=../tucana-out/ REAL_CC=${PWD}/toolchains/llvm-Snapdragon_LLVM_for_Android_8.0/prebuilt/linux-x86_64/bin/clang CLANG_TRIPLE=aarch64-linux-gnu- 2>&1 | tee ../kernel.log
Above solution might be the key thing to solving my issues with DTC. i can safely say I never once put the ARCH= and SUBARCH= in DTC_EXT= this may have solved it.
As for the rom building itself, I may have to use AOSP over QAEP, due to the whole SDCLANG issue
Link:
https://developer.qualcomm.com/forum/qdn-forums/software/snapdragon-llvm-compiler-android/68403#comment-18264
Those issues are all related to QAEP, I have less problems with AOSP.
Just don't have a device tree for it. that is why I need DTC working perfectly.
For what it is worth, tried looking for a default device tree that was not based off lineage-os. No luck thus far.
But isn't the device tree what's in arch/arm64/boot/dts/qcom/ in the kernel sources?
There's even a Makefile there where you can see CONFIG_BUILD_ARM64_DT_OVERLAY in use
b100dian said:
But isn't the device tree what's in arch/arm64/boot/dts/qcom/ in the kernel sources?
There's even a Makefile there where you can see CONFIG_BUILD_ARM64_DT_OVERLAY in use
Click to expand...
Click to collapse
Spoiler: Reply
Lmao holy ****, I think I know where I stuffed up. I was navigating extract only at boot. Did not go further then boot directory lol. I'll test now with navigating to qcom. Thanks for the tip.
***UPDATE***
Managed to work out why all these problems are occurring.
Okay so from what I understand, there is 2 building methods.
1. Using terminal and using export commands.
2. build.config file
To test the theory to make sure that indeed there is a double up on builds, I decided to put the whole Micode Mi 9 export guide into the build.config that I have (made a backup of it) after executing it did indeed show errors.
so then I investigate how it is failing and it came to my attention that the export method for the make defconfig and then build is seperate for build.config.
for the kbuild, we issue the environment variables for example, these below.
When I say its Kbuild variables, it could be clang or gcc. Not 100% sure though.
Spoiler: Kbuild Environment Variables
# PRE_DEFCONFIG_CMDS
# Command evaluated before `make defconfig`
#
# POST_DEFCONFIG_CMDS
# Command evaluated after `make defconfig` and before `make`.
#
# POST_KERNEL_BUILD_CMDS
# Command evaluated after `make`.
#
# EXTRA_CMDS
# Command evaluated after building and installing kernel and modules.
#
# DIST_CMDS
# Command evaluated after copying files to DIST_DIR
#
# VENDOR_RAMDISK_CMDS
# When building vendor boot image, VENDOR_RAMDISK_CMDS enables the build
# config file to specify command(s) for further altering the prebuilt vendor
# ramdisk binary. For example, the build config file could add firmware files
# on the vendor ramdisk (lib/firmware) for testing purposes.
Spoiler: extra-info
and instead of having "make" command by itself I put it for example in PRE_DEFCONFIG_CMDS="make <code>*
and it will add in REAL_CC, etc to the defconfig and upon making the build it should inturn do the rest.
Does anyone know the environment variable for kernel build so i can then put in the make command for the build and include REAL_CC=, etc. I managed to get the first make command for the defconfig working.
PRE_DEFCONFIG_CMDS="make O=out/android-4.14 REAL_CC=${PWD}/prebuilts-master/ndk/toolchains/llvm-Snapdragon_LLVM_for_Android_8.0/prebuilt/linux-x86_64/bin/clang CLANG_TRIPLE=aarch64-linux-gnu- tucana_user_defconfig"
I just now have to get the below command working. not sure what environment variable to put it under.
make -j$(nproc) O=out/android-4.14 REAL_CC=${PWD}/prebuilts-master/ndk/toolchains/llvm-Snapdragon_LLVM_for_Android_8.0/prebuilt/linux-x86_64/bin/clang CLANG_TRIPLE=aarch64-linux-gnu- 2>&1 | tee kernel.log
Found it!
By searching in build directory then build.sh file.
and *_setup_env.sh* has settings as well that will help.
***********************************
export MAKE_ARGS=$*
Could be useful
CC_LD_ARG
***********************************
echo "========================================================"
echo " Building kernel"
set -x
(cd ${OUT_DIR} && make O=${OUT_DIR} ${CC_LD_ARG} ${MAKE_ARGS})
set +x
***Linux update script***
GitHub - android-linux-stable/script: A script to help with merging linux-stable into your own repository
A script to help with merging linux-stable into your own repository - GitHub - android-linux-stable/script: A script to help with merging linux-stable into your own repository
github.com
Better to use CAF for MSM devices.
[REFERENCE] Merge latest CAF Tag in Kernel
Introduction: Hello folks! In this thread I will be guiding you about how you can merge latest CAF tags in your CAF based kernel. Many people who just started compiling the kernels still don't know how to merge a CAF tag because there isn't any...
forum.xda-developers.com
Upon further research on the topic of MAKE_ARGS=$*
from what I now understand and please keep in mind, I am in no way a programmer or good with Linux overall.
I think the $* symbol is a value itself, so from my understanding. If i use lets say $S=Something I have just made an variable that will be used in MAKE_ARGS=$*
If my theory for this is correct, this is how I add in the make variables for the last build command to start the build.
Should be able to do this.
make -j$(nproc) O=out/android-4.14 REAL_CC=${PWD}/prebuilts-master/ndk/toolchains/llvm-Snapdragon_LLVM_for_Android_8.0/prebuilt/linux-x86_64/bin/clang CLANG_TRIPLE=aarch64-linux-gnu- 2>&1 | tee kernel.log
Oh and the config is make commands, found that out as well. you have MAKE and SOONG commands and configs. I so prefer Make variables over Soong. looks easier to read for me.
Found a build config on mi code, sources, issues section
SOURCE
Spoiler: build.config-mi11
#!/bin/bash
export OUT=${PWD}/out
export ARCH=arm64
export SUBARCH=arm64
export TARGET_BUILD_VARIANT=userdebug
#export DTC_EXT=dtc
export CROSS_COMPILE=${PWD}/toolchains/aarch64-linux-android-4.9/bin/aarch64-linux-android-
export KERNEL_DEFCONFIG=venus-qgki_defconfig
#set CONFIG_BUILD_ARM64_DT_OVERLAY=y
#set BUILD_CONFIG=build.config.gki.aarch64
O=$OUT REAL_CC=${PWD}/toolchains/llvm-Snapdragon_LLVM_for_Android_8.0/prebuilt/linux-x86_64/bin/clang CLANG_TRIPLE=aarch64-linux-gnu- ${PWD}/scripts/gki/generate_defconfig.sh $KERNEL_DEFCONFIG
make O=$OUT REAL_CC=${PWD}/toolchains/llvm-Snapdragon_LLVM_for_Android_8.0/prebuilt/linux-x86_64/bin/clang CLANG_TRIPLE=aarch64-linux-gnu- vendor/$KERNEL_DEFCONFIG
make -j$(nproc) O=$OUT REAL_CC=${PWD}/toolchains/llvm-Snapdragon_LLVM_for_Android_8.0/prebuilt/linux-x86_64/bin/clang CLANG_TRIPLE=aarch64-linux-gnu- 2>&1 | tee kernel.log
The build config above is indeed for Mi11, we do not need gsi settings or the 3rd command starting with O=$OUT for Mi note 10 pro
By implementing the changes above, outcome below.
Just now have to resolve kernel issues and this should build with no more issues. it has been a massive adventure.
the errors above are due to remnants of old files from merges that were not undone properly. deleted the devices source and recreated it again. problems are now gone will update on outcome.
I still can not seem to get it completely built. Max Image.gz-dtb file size is 28.5MB so far. and the highest I have gotten. still get VMLINUX issues which tells me just maybe Xiaomi themselves have not updated the build scripts to suite the latest CAF changes.
All speculation at the moment.
Had a look at this area
GitHub - MiCode/kernel_build
Contribute to MiCode/kernel_build development by creating an account on GitHub.
github.com
Turns out, build config for CC9 Pro. does not exist hence why all the problems Using AOSP's.
I believe it needs to be full modified in order to work with Tucana, due to Xiaomi not releasing a build script for it. so another solution would be to modify or try and use another device on the list in the link to be able to build it. not even export using the Mi 9 works so I have to dig deeper on getting a config and build script for it.
Spoiler: Current build.config
ARCH=arm64
SUBARCH=arm64
BRANCH=android-4.14
CLANG_TRIPLE=aarch64-linux-gnu-
CROSS_COMPILE=aarch64-linux-android-
TARGET_BUILD_VARIANT=userdebug
DEFCONFIG=tucana_user_defconfig
SKIP_DEFCONFIG=1
PRE_DEFCONFIG_CMDS="make O=out/android-4.14 REAL_CC=${ROOT_DIR}/prebuilts-master/ndk/toolchains/llvm-Snapdragon_LLVM_for_Android_8.0/prebuilt/linux-x86_64/bin/clang CLANG_TRIPLE=aarch64-linux-gnu- ${DEFCONFIG} && make O=out/android-4.14 menuconfig"
POST_DEFCONFIG_CMDS=""
KERNEL_DIR=.
EXTRA_CMDS=""
HOSTCC=gcc
CC=clang
DTC_EXT=${ROOT_DIR}/prebuilts-master/misc/linux-x86/dtc/dtc
LINUX_GCC_CROSS_COMPILE_PREBUILTS_BIN=prebuilts/gcc/linux-x86/aarch64/aarch64-linux-android-4.9/bin
CLANG_PREBUILT_BIN=prebuilts-master/ndk/toolchains/llvm-Snapdragon_LLVM_for_Android_8.0/prebuilt/linux-x86_64/bin
LZ4_PREBUILTS_BIN=prebuilts-master/misc/linux-x86/lz4
DTC_PREBUILTS_BIN=prebuilts-master/misc/linux-x86/dtc
LIBUFDT_PREBUILTS_BIN=prebuilts-master/misc/linux-x86/libufdt
BUILDTOOLS_PREBUILT_BIN=build/build-tools/path/linux-x86
OUT_DIR=out/android-4.14
FILES="
arch/arm64/boot/Image
arch/arm64/boot/Image.gz
arch/arm64/boot/Image.gz-dtb
arch/arm64/boot/dts/qcom/tucana-sdmmagpie-overlay.dtbo
arch/arm64/boot/dts/qcom/sdmmagpie.dtb
vmlinux
System.map
.config
"
EXT_MODULES="
private/msm-tucana-modules/wlan/qcacld-3.0
"
IN_KERNEL_MODULES=1
STOP_SHIP_TRACEPRINTK=1
The config does not build vmlinux, DTC properly, dtbo.img. Still working on it.
I also did try first doing the standalone export way. Just constantly getting ld error --fix-something and to fix it, by using CC=clang and HOSTCC=gcc
but then you resort to having to use the build.config file. for some reason the standalone export method just does not work with cc and hostcc
and that is using the QQ LLVM/clang 8.0 toolchain with AOSP's GCC while having 2 directories. toolchain(GCC) and toolchains(QQ-LLVM/Clang) with QQ-llvm/clang's build directory inside the devices source root directory as well, according to MiCode Wiki under MSM-4.14.
Tried attaching the last make command to the DTC_EXT= and it fails, still unable to find the build kernel argument to change the command manually. Only have POST_KERNEL_BUILD
No luck at all with the export combo.
As for the build.config.
I get a build using AOSP'S GCC and QQ-LLVM/CLANG 8.0
But it is not complete.
I also use the build, prebuilts, prebuilts-master from AOSP's 4.14 Common kernel repo.
the reason for this, is because Xiaomi have not released kernel build files for the device. from what I have found.

General Intro to Cryptography By a Noob

Hello Friends~!
I have built information trees, to teach you about these tools. I was originally going to package them all together, but in my efforts of doing so I realize that the process was getting too convoluted, when there is already an excellent "package manager" called Chocolatey that will allow us to Download all These and more, with "wrapped" permissions, meaning you don't have to go into settings, and program environment variables for Every Single Program you want to install. I will make a separate thread on Chocolatey and explain better what it is, and how to use it
READ THE WARNING IN THE THREAD BELOW BEFORE DOWNLOADING OPENSSL
TOOLS FOR ENCRYPTION AND BUILDING APPLICATIONS, SOURCES
Spoiler: OPENSSL
Please read the WARNING on this Thread about CRYPTOGRAPHY before downloading...
Spoiler: THREAD
How to Make and Sign a Driver and Certificate: Intro To Encryption
========================================= HOW TO MAKE AND SIGN, A DRIVER AND CERTIFICATE: INTRO TO ENCRYPTION~! ========================================= Today we are going to get our feet a little wet in Cryptography~! Why would I need this...
forum.xda-developers.com
Spoiler: DOWNLOAD
GitHub - openssl/openssl: TLS/SSL and crypto library
TLS/SSL and crypto library. Contribute to openssl/openssl development by creating an account on GitHub.
github.com
You need to click on Clone, then download zip.
Spoiler: LIKE THIS
{
"lightbox_close": "Close",
"lightbox_next": "Next",
"lightbox_previous": "Previous",
"lightbox_error": "The requested content cannot be loaded. Please try again later.",
"lightbox_start_slideshow": "Start slideshow",
"lightbox_stop_slideshow": "Stop slideshow",
"lightbox_full_screen": "Full screen",
"lightbox_thumbnails": "Thumbnails",
"lightbox_download": "Download",
"lightbox_share": "Share",
"lightbox_zoom": "Zoom",
"lightbox_new_window": "New window",
"lightbox_toggle_sidebar": "Toggle sidebar"
}
Spoiler: WHAT IS OPEN SSL
Spoiler: README
Welcome to the OpenSSL Project
www.openssl.org
OpenSSL is a robust, commercial-grade, full-featured Open Source Toolkit
for the Transport Layer Security (TLS) protocol formerly known as the
Secure Sockets Layer (SSL) protocol. The protocol implementation is based
on a full-strength general purpose cryptographic library, which can also
be used stand-alone.
OpenSSL is descended from the SSLeay library developed by Eric A. Young
and Tim J. Hudson.
The official Home Page of the OpenSSL Project is [www.openssl.org].
Spoiler: OVERVIEW
The OpenSSL toolkit includes:
- **libssl**
an implementation of all TLS protocol versions up to TLSv1.3 ([RFC 8446]).
- **libcrypto**
a full-strength general purpose cryptographic library. It constitutes the
basis of the TLS implementation, but can also be used independently.
- **openssl**
the OpenSSL command line tool, a swiss army knife for cryptographic tasks,
testing and analyzing. It can be used for
- creation of key parameters
- creation of X.509 certificates, CSRs and CRLs
- calculation of message digests
- encryption and decryption
- SSL/TLS client and server tests
- handling of S/MIME signed or encrypted mail
- and more...
Spoiler: FOR PRODUCTION USE
Source code tarballs of the official releases can be downloaded from
[www.openssl.org/source](https://www.openssl.org/source).
The OpenSSL project does not distribute the toolkit in binary form.
However, for a large variety of operating systems precompiled versions
of the OpenSSL toolkit are available. In particular on Linux and other
Unix operating systems it is normally recommended to link against the
precompiled shared libraries provided by the distributor or vendor.
Spoiler: FOR TESTING AND DEVELOPMENT
Although testing and development could in theory also be done using
the source tarballs, having a local copy of the git repository with
the entire project history gives you much more insight into the
code base.
The official OpenSSL Git Repository is located at [git.openssl.org].
There is a GitHub mirror of the repository at [github.com/openssl/openssl],
which is updated automatically from the former on every commit.
A local copy of the Git Repository can be obtained by cloning it from
the original OpenSSL repository using
git clone git://git.openssl.org/openssl.git
or from the GitHub mirror using
git clone https://github.com/openssl/openssl.git
If you intend to contribute to OpenSSL, either to fix bugs or contribute
new features, you need to fork the OpenSSL repository openssl/openssl on
GitHub and clone your public fork instead.
git clone https://github.com/yourname/openssl.git
This is necessary, because all development of OpenSSL nowadays is done via
GitHub pull requests. For more details, see [Contributing](#contributing).
Spoiler: BUILD AND INSTALL
After obtaining the Source, have a look at the [INSTALL](INSTALL.md) file for
detailed instructions about building and installing OpenSSL. For some
platforms, the installation instructions are amended by a platform specific
document.
* [Notes for UNIX-like platforms](NOTES-UNIX.md)
* [Notes for Android platforms](NOTES-ANDROID.md)
* [Notes for Windows platforms](NOTES-WINDOWS.md)
* [Notes for the DOS platform with DJGPP](NOTES-DJGPP.md)
* [Notes for the OpenVMS platform](NOTES-VMS.md)
* [Notes on Perl](NOTES-PERL.md)
* [Notes on Valgrind](NOTES-VALGRIND.md)
Specific notes on upgrading to OpenSSL 3.0 from previous versions, as well as
known issues are available on the [OpenSSL 3.0 Wiki] page.
Spoiler: DOCUMENTATION
Manual Pages
------------
The manual pages for the master branch and all current stable releases are
available online.
- [OpenSSL master](https://www.openssl.org/docs/manmaster)
- [OpenSSL 1.1.1](https://www.openssl.org/docs/man1.1.1)
Wiki
----
There is a Wiki at [wiki.openssl.org] which is currently not very active.
It contains a lot of useful information, not all of which is up to date.
License
=======
OpenSSL is licensed under the Apache License 2.0, which means that
you are free to get and use it for commercial and non-commercial
purposes as long as you fulfill its conditions.
See the [LICENSE.txt](LICENSE.txt) file for more details.
Support
=======
There are various ways to get in touch. The correct channel depends on
your requirement. see the [SUPPORT](SUPPORT.md) file for more details.
Contributing
============
If you are interested and willing to contribute to the OpenSSL project,
please take a look at the [CONTRIBUTING](CONTRIBUTING.md) file.
Legalities
==========
A number of nations restrict the use or export of cryptography. If you are
potentially subject to such restrictions you should seek legal advice before
attempting to develop or distribute cryptographic code.
Copyright
=========
Copyright (c) 1998-2021 The OpenSSL Project
Copyright (c) 1995-1998 Eric A. Young, Tim J. Hudson
All rights reserved.
<!-- Links -->
[www.openssl.org]:
<https://www.openssl.org>
"OpenSSL Homepage"
[git.openssl.org]:
<https://git.openssl.org>
"OpenSSL Git Repository"
[git.openssl.org]:
<https://git.openssl.org>
"OpenSSL Git Repository"
[github.com/openssl/openssl]:
<https://github.com/openssl/openssl>
"OpenSSL GitHub Mirror"
[wiki.openssl.org]:
<https://wiki.openssl.org>
"OpenSSL Wiki"
[OpenSSL 3.0 Wiki]:
<https://wiki.openssl.org/index.php/OpenSSL_3.0>
"OpenSSL 3.0 Wiki"
[RFC 8446]:
<https://tools.ietf.org/html/rfc8446>
<!-- Logos and Badges -->
[openssl logo]:
doc/images/openssl.svg
"OpenSSL Logo"
[github actions ci badge]:
<https://github.com/openssl/openssl/workflows/GitHub CI/badge.svg>
"GitHub Actions CI Status"
[github actions ci]:
<https://github.com/openssl/openssl/actions?query=workflow:"GitHub+CI">
"GitHub Actions CI"
[appveyor badge]:
<https://ci.appveyor.com/api/projects/status/8e10o7xfrg73v98f/branch/master?svg=true>
"AppVeyor Build Status"
[appveyor jobs]:
<https://ci.appveyor.com/project/openssl/openssl/branch/master>
"AppVeyor Jobs"
Spoiler: INSTALLATION AND EXPLANATION OF SYSTEMS
This document describes installation on all supported operating
systems (the Unix/Linux family, including macOS), OpenVMS,
and Windows).
Spoiler: TABLE OF CONTENTS
Spoiler: PREREQUISITES
To install OpenSSL, you will need:
* A "make" implementation
* Perl 5 with core modules (please read [NOTES-PERL.md](NOTES-PERL.md))
* The Perl module `Text::Template` (please read [NOTES-PERL.md](NOTES-PERL.md))
* an ANSI C compiler
* a development environment in the form of development libraries and C
header files
* a supported operating system
Spoiler: NOTATIONAL CONVENTIONS
Throughout this document, we use the following conventions.
Spoiler: COMMANDS
Any line starting with a dollar sign is a command line.
$ command
The dollar sign indicates the shell prompt and is not to be entered as
part of the command.
Spoiler: CHOICES
Several words in curly braces separated by pipe characters indicate a
**mandatory choice**, to be replaced with one of the given words.
Spoiler: EXAMPLE
For example, the line
$ echo { WORD1 | WORD2 | WORD3 }
represents one of the following three commands
$ echo WORD1
- or -
$ echo WORD2
- or -
$ echo WORD3
One or several words in square brackets separated by pipe characters
denote an **optional choice**. It is similar to the mandatory choice,
but it can also be omitted entirely.
Spoiler: EXAMPLE
So the line
$ echo [ WORD1 | WORD2 | WORD3 ]
represents one of the four commands
$ echo WORD1
- or -
$ echo WORD2
- or -
$ echo WORD3
- or -
$ echo
Spoiler: ARGUMENTS
**Optional Arguments** are enclosed in square brackets.
[option...]
A trailing ellipsis means that more than one could be specified.
Spoiler: QUICK INSTALLATION GUIDE
If you just want to get OpenSSL installed without bothering too much
about the details, here is the short version of how to build and install
OpenSSL. If any of the following steps fails, please consult the
[Installation in Detail](#installation-steps-in-detail) section below.
Spoiler: BUILDING OPENSSL
Use the following commands to configure, build and test OpenSSL.
The testing is optional, but recommended if you intend to install
OpenSSL for production use.
Spoiler: UNIX LINUX MAC
$ ./Configure
$ make
$ make test
Spoiler: OPENVMS
Use the following commands to build OpenSSL:
$ perl Configure
$ mms
$ mms test
Spoiler: WINDOWS
If you are using Visual Studio, open a Developer Command Prompt and
issue the following commands to build OpenSSL.
$ perl Configure
$ nmake
$ nmake test
As mentioned in the [Choices](#choices) section, you need to pick one
of the four Configure targets in the first command.
Most likely you will be using the `VC-WIN64A` target for 64bit Windows
binaries (AMD64) or `VC-WIN32` for 32bit Windows binaries (X86).
The other two options are `VC-WIN64I` (Intel IA64, Itanium) and
`VC-CE` (Windows CE) are rather uncommon nowadays.
Spoiler: INSTALLING OPENSSL
The following commands will install OpenSSL to a default system location.
**Danger Zone:** even if you are impatient, please read the following two
paragraphs carefully before you install OpenSSL.
For security reasons the default system location is by default not writable
for unprivileged users. So for the final installation step administrative
privileges are required. The default system location and the procedure to
obtain administrative privileges depends on the operating system.
It is recommended to compile and test OpenSSL with normal user privileges
and use administrative privileges only for the final installation step.
Spoiler: SYSTEMS WITH OPENSSL PREINSTALLED
On some platforms OpenSSL is preinstalled as part of the Operating System.
In this case it is highly recommended not to overwrite the system versions,
because other applications or libraries might depend on it.
To avoid breaking other applications, install your copy of OpenSSL to a
[different location](#installing-to-a-different-location) which is not in
the global search path for system libraries.
Finally, if you plan on using the FIPS module, you need to read the
[Post-installation Notes](#post-installation-notes) further down.
Spoiler: UNIX LINUX MAC
Depending on your distribution, you need to run the following command as
root user or prepend `sudo` to the command:
$ make install
By default, OpenSSL will be installed to
/usr/local
More precisely, the files will be installed into the subdirectories
/usr/local/bin
/usr/local/lib
/usr/local/include
depending on the file type, as it is custom on Unix-like operating systems.
Spoiler: OPENVMS
Use the following command to install OpenSSL.
$ mms install
By default, OpenSSL will be installed to
SYS$COMMON:[OPENSSL]
Spoiler: WINDOWS
Spoiler: USING VISUAL STUDIO
If you are using Visual Studio, open the Developer Command Prompt _elevated_
and issue the following command.
$ nmake install
The easiest way to elevate the Command Prompt is to press and hold down
the both the `<CTRL>` and `<SHIFT>` key while clicking the menu item in the
task menu.
The default installation location is
C:\Program Files\OpenSSL
for native binaries, or
C:\Program Files (x86)\OpenSSL
for 32bit binaries on 64bit Windows (WOW64).
Spoiler: INSTALLING TO A DIFFERENT LOCATION
To install OpenSSL to a different location (for example into your home
directory for testing purposes) run `Configure` as shown in the following
examples.
The options `--prefix` and `--openssldir` are explained in further detail in
[Directories](#directories) below, and the values used here are mere examples.
Spoiler: UNIX OPENVMS
On Unix:
$ ./Configure --prefix=/opt/openssl --openssldir=/usr/local/ssl
On OpenVMS:
$ perl Configure --prefix=PROGRAM:[INSTALLS] --openssldir=SYS$MANAGER:[OPENSSL]
Note: if you do add options to the configuration command, please make sure
you've read more than just this Quick Start, such as relevant `NOTES-*` files,
the options outline below, as configuration options may change the outcome
in otherwise unexpected ways.
Spoiler: CONFIGURATION OPTIONS
There are several options to `./Configure` to customize the build (note that
for Windows, the defaults for `--prefix` and `--openssldir` depend on what
configuration is used and what Windows implementation OpenSSL is built on.
For more information, see the [Notes for Windows platforms](NOTES-WINDOWS.md).
Spoiler: API LEVEL
Spoiler: API=X.Y[.Z]
--api=x.y[.z]
Build the OpenSSL libraries to support the API for the specified version.
If [no-deprecated](#no-deprecated) is also given, don't build with support
for deprecated APIs in or below the specified version number. For example,
adding
Spoiler: API=1.1.0 NO-DEPRECATED
--api=1.1.0 no-deprecated
will remove support for all APIs that were deprecated in OpenSSL version
1.1.0 or below. This is a rather specialized option for developers.
If you just intend to remove all deprecated APIs up to the current version
entirely, just specify [no-deprecated](#no-deprecated).
If `--api` isn't given, it defaults to the current (minor) OpenSSL version.
Spoiler: CROSS COMPILE PREFIX
--cross-compile-prefix=<PREFIX>
The `<PREFIX>` to include in front of commands for your toolchain.
It is likely to have to end with dash, e.g. `a-b-c-` would invoke GNU compiler
as `a-b-c-gcc`, etc. Unfortunately cross-compiling is too case-specific to put
together one-size-fits-all instructions. You might have to pass more flags or
set up environment variables to actually make it work. Android and iOS cases
are discussed in corresponding `Configurations/15-*.conf` files. But there are
cases when this option alone is sufficient. For example to build the mingw64
target on Linux `--cross-compile-prefix=x86_64-w64-mingw32-` works. Naturally
provided that mingw packages are installed. Today Debian and Ubuntu users
have option to install a number of prepackaged cross-compilers along with
corresponding run-time and development packages for "alien" hardware. To give
another example `--cross-compile-prefix=mipsel-linux-gnu-` suffices in such
case.
For cross compilation, you must [configure manually](#manual-configuration).
Also, note that `--openssldir` refers to target's file system, not one you are
building on.
Spoiler: BUILD TYPE
--debug
Build OpenSSL with debugging symbols and zero optimization level.
--release
Build OpenSSL without debugging symbols. This is the default.
Spoiler: DIRECTORIES
Spoiler: LIBDIR
--libdir=DIR
The name of the directory under the top of the installation directory tree
(see the `--prefix` option) where libraries will be installed. By default
this is `lib`. Note that on Windows only static libraries (`*.lib`) will
be stored in this location. Shared libraries (`*.dll`) will always be
installed to the `bin` directory.
Some build targets have a multilib postfix set in the build configuration.
For these targets the default libdir is `lib<multilib-postfix>`. Please use
`--libdir=lib` to override the libdir if adding the postfix is undesirable.
Spoiler: OPENSSLDIR
--openssldir=DIR
Directory for OpenSSL configuration files, and also the default certificate
and key store. Defaults are:
Unix: /usr/local/ssl
Windows: C:\Program Files\Common Files\SSL
OpenVMS: SYS$COMMON:[OPENSSL-COMMON]
For 32bit Windows applications on Windows 64bit (WOW64), always replace
`C:\Program Files` by `C:\Program Files (x86)`.
Spoiler: PREFIX
--prefix=DIR
The top of the installation directory tree. Defaults are:
Unix: /usr/local
Windows: C:\Program Files\OpenSSL
OpenVMS: SYS$COMMON:[OPENSSL]
Spoiler: COMPILER WARNINGS
--strict-warnings
This is a developer flag that switches on various compiler options recommended
for OpenSSL development. It only works when using gcc or clang as the compiler.
If you are developing a patch for OpenSSL then it is recommended that you use
this option where possible.
Spoiler: ZLIB FLAGS
Spoiler: WITH-ZLIB-INCLUDE
--with-zlib-include=DIR
The directory for the location of the zlib include file. This option is only
necessary if [zlib](#zlib) is used and the include file is not
already on the system include path.
Spoiler: WITH-ZLIB-LIB
--with-zlib-lib=LIB
Spoiler: UNIX
**On Unix**: this is the directory containing the zlib library.
If not provided the system library path will be used.
Spoiler: WINDOWS
**On Windows:** this is the filename of the zlib library (with or
without a path). This flag must be provided if the
[zlib-dynamic](#zlib-dynamic) option is not also used. If `zlib-dynamic` is used
then this flag is optional and defaults to `ZLIB1` if not provided.
Spoiler: VMS
**On VMS:** this is the filename of the zlib library (with or without a path).
This flag is optional and if not provided then `GNV$LIBZSHR`, `GNV$LIBZSHR32`
or `GNV$LIBZSHR64` is used by default depending on the pointer size chosen.
Spoiler: SEEDING THE RANDOM NUMBER GENERATOR
--with-rand-seed=seed1[,seed2,...]
A comma separated list of seeding methods which will be tried by OpenSSL
in order to obtain random input (a.k.a "entropy") for seeding its
cryptographically secure random number generator (CSPRNG).
The current seeding methods are:
Spoiler: OS
### os
Use a trusted operating system entropy source.
This is the default method if such an entropy source exists.
Spoiler: GETRANDOM
### getrandom
Use the [getrandom(2)][man-getrandom] or equivalent system call.
[man-getrandom]: http://man7.org/linux/man-pages/man2/getrandom.2.html
Spoiler: DEVRANDOM
### devrandom
Use the first device from the `DEVRANDOM` list which can be opened to read
random bytes. The `DEVRANDOM` preprocessor constant expands to
"/dev/urandom","/dev/random","/dev/srandom"
on most unix-ish operating systems.
Spoiler: EGD
### egd
Check for an entropy generating daemon.
This source is ignored by the FIPS provider.
Spoiler: RDCPU
### rdcpu
Use the `RDSEED` or `RDRAND` command on x86 or `RNDRRS` command on aarch64
if provided by the CPU.
Spoiler: LIBRANDOM
### librandom
Use librandom (not implemented yet).
This source is ignored by the FIPS provider.
Spoiler: NONE
### none
Disable automatic seeding. This is the default on some operating systems where
no suitable entropy source exists, or no support for it is implemented yet.
This option is ignored by the FIPS provider.
For more information, see the section [Notes on random number generation][rng]
at the end of this document.
[rng]: #notes-on-random-number-generation
Spoiler: SETTING THE FIPS HMAC KEY
Spoiler: FIPS-KEY=VALUE
--fips-key=value
As part of its self-test validation, the FIPS module must verify itself
by performing a SHA-256 HMAC computation on itself. The default key is
the SHA256 value of "the holy handgrenade of antioch" and is sufficient
for meeting the FIPS requirements.
To change the key to a different value, use this flag. The value should
be a hex string no more than 64 characters.
Spoiler: ENABLE AND DISABLE FEATURES
Feature options always come in pairs, an option to enable feature
`xxxx`, and an option to disable it:
[ enable-xxxx | no-xxxx ]
Whether a feature is enabled or disabled by default, depends on the feature.
In the following list, always the non-default variant is documented: if
feature `xxxx` is disabled by default then `enable-xxxx` is documented and
if feature `xxxx` is enabled by default then `no-xxxx` is documented.
Spoiler: NO-AFALGENG
### no-afalgeng
Don't build the AFALG engine.
This option will be forced on a platform that does not support AFALG.
Spoiler: ENABLE-KTLS
### enable-ktls
Build with Kernel TLS support.
This option will enable the use of the Kernel TLS data-path, which can improve
performance and allow for the use of sendfile and splice system calls on
TLS sockets. The Kernel may use TLS accelerators if any are available on the
system. This option will be forced off on systems that do not support the
Kernel TLS data-path.
Spoiler: ENABLE-ASAN
### enable-asan
Build with the Address sanitiser.
This is a developer option only. It may not work on all platforms and should
never be used in production environments. It will only work when used with
gcc or clang and should be used in conjunction with the [no-shared](#no-shared)
option.
Spoiler: ENABLE-ACVP-TESTS
### enable-acvp-tests
Build support for Automated Cryptographic Validation Protocol (ACVP)
tests.
This is required for FIPS validation purposes. Certain ACVP tests require
access to algorithm internals that are not normally accessible.
Additional information related to ACVP can be found at
<https://github.com/usnistgov/ACVP>.
Spoiler: NO-ASM
### no-asm
Do not use assembler code.
This should be viewed as debugging/troubleshooting option rather than for
production use. On some platforms a small amount of assembler code may still
be used even with this option.
Spoiler: NO-ASYNC
### no-async
Do not build support for async operations.
Spoiler: NO-AUTOALGINIT
### no-autoalginit
Don't automatically load all supported ciphers and digests.
Typically OpenSSL will make available all of its supported ciphers and digests.
For a statically linked application this may be undesirable if small executable
size is an objective. This only affects libcrypto. Ciphers and digests will
have to be loaded manually using `EVP_add_cipher()` and `EVP_add_digest()`
if this option is used. This option will force a non-shared build.
Spoiler: NO-AUTOERRINIT
### no-autoerrinit
Don't automatically load all libcrypto/libssl error strings.
Typically OpenSSL will automatically load human readable error strings. For a
statically linked application this may be undesirable if small executable size
is an objective.
Spoiler: NO-AUTOLOAD-CONFIG
### no-autoload-config
Don't automatically load the default `openssl.cnf` file.
Typically OpenSSL will automatically load a system config file which configures
default SSL options.
Spoiler: ENABLE-BUILDTEST-C++
### enable-buildtest-c++
While testing, generate C++ buildtest files that simply check that the public
OpenSSL header files are usable standalone with C++.
Enabling this option demands extra care. For any compiler flag given directly
as configuration option, you must ensure that it's valid for both the C and
the C++ compiler. If not, the C++ build test will most likely break. As an
alternative, you can use the language specific variables, `CFLAGS` and `CXXFLAGS`.
Spoiler: BANNER=TEXT
### --banner=text
Use the specified text instead of the default banner at the end of
configuration.
Spoiler: W
### --w
On platforms where the choice of 32-bit or 64-bit architecture
is not explicitly specified, `Configure` will print a warning
message and wait for a few seconds to let you interrupt the
configuration. Using this flag skips the wait.
Spoiler: NO-BULK
### no-bulk
Build only some minimal set of features.
This is a developer option used internally for CI build tests of the project.
Spoiler: NO-CACHED-FETCH
### no-cached-fetch
Never cache algorithms when they are fetched from a provider. Normally, a
provider indicates if the algorithms it supplies can be cached or not. Using
this option will reduce run-time memory usage but it also introduces a
significant performance penalty. This option is primarily designed to help
with detecting incorrect reference counting.
Spoiler: NO-CAPIENG
### no-capieng
Don't build the CAPI engine.
This option will be forced if on a platform that does not support CAPI.
Spoiler: NO-CMP
### no-cmp
Don't build support for Certificate Management Protocol (CMP)
and Certificate Request Message Format (CRMF).
Spoiler: NO-CMS
### no-cms
Don't build support for Cryptographic Message Syntax (CMS).
Spoiler: NO-COMP
### no-comp
Don't build support for SSL/TLS compression.
If this option is enabled (the default), then compression will only work if
the zlib or `zlib-dynamic` options are also chosen.
Spoiler: ENABLE-CRYPTO-MDEBUG
### enable-crypto-mdebug
This now only enables the `failed-malloc` feature.
Spoiler: ENABLE-CRYPTO-MDEBUG-BACKTRACE
### enable-crypto-mdebug-backtrace
This is a no-op; the project uses the compiler's address/leak sanitizer instead.
Spoiler: NO-CT
### no-ct
Don't build support for Certificate Transparency (CT).
Spoiler: NO-DEPRECATED
### no-deprecated
Don't build with support for deprecated APIs up until and including the version
given with `--api` (or the current version, if `--api` wasn't specified).
Spoiler: NO-DGRAM
### no-dgram
Don't build support for datagram based BIOs.
Selecting this option will also force the disabling of DTLS.
Spoiler: NO-DSO
### no-dso
Don't build support for loading Dynamic Shared Objects (DSO)
Spoiler: ENABLE-DEVCRYPTOENG
### enable-devcryptoeng
Build the `/dev/crypto` engine.
This option is automatically selected on the BSD platform, in which case it can
be disabled with `no-devcryptoeng`.
Spoiler: NO-DYNAMIC-ENGINE
### no-dynamic-engine
Don't build the dynamically loaded engines.
This only has an effect in a shared build.
Spoiler: NO-EC
### no-ec
Don't build support for Elliptic Curves.
Spoiler: NO-EC2M
### no-ec2m
Don't build support for binary Elliptic Curves
Spoiler: ENABLE-EC_NISTP_64_GCC_128
### enable-ec_nistp_64_gcc_128
Enable support for optimised implementations of some commonly used NIST
elliptic curves.
This option is only supported on platforms:
- with little-endian storage of non-byte types
- that tolerate misaligned memory references
- where the compiler:
- supports the non-standard type `__uint128_t`
- defines the built-in macro `__SIZEOF_INT128__`
Spoiler: ENABLE-EGD
### enable-egd
Build support for gathering entropy from the Entropy Gathering Daemon (EGD).
Spoiler: NO-ENGINE
### no-engine
Don't build support for loading engines.
Spoiler: NO-ERR
### no-err
Don't compile in any error strings.
Spoiler: ENABLE-EXTERNAL-TESTS
### enable-external-tests
Enable building of integration with external test suites.
This is a developer option and may not work on all platforms. The following
external test suites are currently supported:
- GOST engine test suite
- Python PYCA/Cryptography test suite
- krb5 test suite
See the file [test/README-external.md](test/README-external.md)
for further details.
Spoiler: NO-FILENAMES
### no-filenames
Don't compile in filename and line number information (e.g. for errors and
memory allocation).
Spoiler: ENABLE-FIPS
### enable-fips
Build (and install) the FIPS provider
Spoiler: NO-FIPS-SECURITYCHECKS
### no-fips-securitychecks
Don't perform FIPS module run-time checks related to enforcement of security
parameters such as minimum security strength of keys.
Spoiler: ENABLE-FUZZ-LIBFUZZER, ENABLE-FUZZ-AFL
### enable-fuzz-libfuzzer, enable-fuzz-afl
Build with support for fuzzing using either libfuzzer or AFL.
These are developer options only. They may not work on all platforms and
should never be used in production environments.
See the file [fuzz/README.md](fuzz/README.md) for further details.
Spoiler: NO-GOST
### no-gost
Don't build support for GOST based ciphersuites.
Note that if this feature is enabled then GOST ciphersuites are only available
if the GOST algorithms are also available through loading an externally supplied
engine.
Spoiler: NO-LEGACY
### no-legacy
Don't build the legacy provider.
Disabling this also disables the legacy algorithms: MD2 (already disabled by default).
Spoiler: NO-MAKEDEPEND
### no-makedepend
Don't generate dependencies.
Spoiler: NO-MODULE
### no-module
Don't build any dynamically loadable engines.
This also implies `no-dynamic-engine`.
Spoiler: NO-MULTIBLOCK
### no-multiblock
Don't build support for writing multiple records in one go in libssl
Note: this is a different capability to the pipelining functionality.
Spoiler: NO-NEXTPROTONEG
### no-nextprotoneg
Don't build support for the Next Protocol Negotiation (NPN) TLS extension.
Spoiler: NO-OCSP
### no-ocsp
Don't build support for Online Certificate Status Protocol (OCSP).
Spoiler: NO-PADLOCKENG
### no-padlockeng
Don't build the padlock engine.
Spoiler: NO-HW-PADLOCK
### no-hw-padlock
As synonym for `no-padlockeng`. Deprecated and should not be used.
Spoiler: NO-PIC
### no-pic
Don't build with support for Position Independent Code.
Spoiler: NO-PINSHARED
### no-pinshared
Don't pin the shared libraries.
By default OpenSSL will attempt to stay in memory until the process exits.
This is so that libcrypto and libssl can be properly cleaned up automatically
via an `atexit()` handler. The handler is registered by libcrypto and cleans
up both libraries. On some platforms the `atexit()` handler will run on unload of
libcrypto (if it has been dynamically loaded) rather than at process exit. This
option can be used to stop OpenSSL from attempting to stay in memory until the
process exits. This could lead to crashes if either libcrypto or libssl have
already been unloaded at the point that the atexit handler is invoked, e.g. on a
platform which calls `atexit()` on unload of the library, and libssl is unloaded
before libcrypto then a crash is likely to happen. Applications can suppress
running of the `atexit()` handler at run time by using the
`OPENSSL_INIT_NO_ATEXIT` option to `OPENSSL_init_crypto()`.
See the man page for it for further details.
Spoiler: NO-POSIX-IO
### no-posix-io
Don't use POSIX IO capabilities.
Spoiler: NO-PSK
### no-psk
Don't build support for Pre-Shared Key based ciphersuites.
Spoiler: NO-RDRAND
### no-rdrand
Don't use hardware RDRAND capabilities.
Spoiler: NO-RFC3779
### no-rfc3779
Don't build support for RFC3779, "X.509 Extensions for IP Addresses and
AS Identifiers".
Spoiler: SCTP
### sctp
Build support for Stream Control Transmission Protocol (SCTP).
Spoiler: NO-SHARED
### no-shared
Do not create shared libraries, only static ones.
See [Notes on shared libraries](#notes-on-shared-libraries) below.
Spoiler: NO-SOCK
### no-sock
Don't build support for socket BIOs.
Spoiler: NO-SRP
### no-srp
Don't build support for Secure Remote Password (SRP) protocol or
SRP based ciphersuites.
Spoiler: NO-SRTP
### no-srtp
Don't build Secure Real-Time Transport Protocol (SRTP) support.
Spoiler: NO-SSE2
### no-sse2
Exclude SSE2 code paths from 32-bit x86 assembly modules.
Normally SSE2 extension is detected at run-time, but the decision whether or not
the machine code will be executed is taken solely on CPU capability vector. This
means that if you happen to run OS kernel which does not support SSE2 extension
on Intel P4 processor, then your application might be exposed to "illegal
instruction" exception. There might be a way to enable support in kernel, e.g.
FreeBSD kernel can be compiled with `CPU_ENABLE_SSE`, and there is a way to
disengage SSE2 code paths upon application start-up, but if you aim for wider
"audience" running such kernel, consider `no-sse2`. Both the `386` and `no-asm`
options imply `no-sse2`.
Spoiler: NO-SSL-TRACE
### no-ssl-trace
Don't build with SSL Trace capabilities.
This removes the `-trace` option from `s_client` and `s_server`, and omits the
`SSL_trace()` function from libssl.
Disabling `ssl-trace` may provide a small reduction in libssl binary size.
Spoiler: NO-STATIC-ENGINE
### no-static-engine
Don't build the statically linked engines.
This only has an impact when not built "shared".
Spoiler: NO-STDIO
### no-stdio
Don't use anything from the C header file `stdio.h` that makes use of the `FILE`
type. Only libcrypto and libssl can be built in this way. Using this option will
suppress building the command line applications. Additionally, since the OpenSSL
tests also use the command line applications, the tests will also be skipped.
Spoiler: NO-TESTS
### no-tests
Don't build test programs or run any tests.
Spoiler: NO-THREADS
### no-threads
Don't build with support for multi-threaded applications.
Spoiler: THREADS
### threads
Build with support for multi-threaded applications. Most platforms will enable
this by default. However, if on a platform where this is not the case then this
will usually require additional system-dependent options!
See [Notes on multi-threading](#notes-on-multi-threading) below.
Spoiler: ENABLE-TRACE
### enable-trace
Build with support for the integrated tracing api.
See manual pages OSSL_trace_set_channel(3) and OSSL_trace_enabled(3) for details.
Spoiler: NO-TS
### no-ts
Don't build Time Stamping (TS) Authority support.
Spoiler: ENABLE-UBSAN
### enable-ubsan
Build with the Undefined Behaviour sanitiser (UBSAN).
This is a developer option only. It may not work on all platforms and should
never be used in production environments. It will only work when used with
gcc or clang and should be used in conjunction with the `-DPEDANTIC` option
(or the `--strict-warnings` option).
Spoiler: NO-UI-CONSOLE
### no-ui-console
Don't build with the User Interface (UI) console method
The User Interface console method enables text based console prompts.
Spoiler: ENABLE-UNIT-TEST
### enable-unit-test
Enable additional unit test APIs.
This should not typically be used in production deployments.
Spoiler: NO-UPLINK
### no-uplink
Don't build support for UPLINK interface.
Spoiler: ENABLE-WEAK-SSL-CIPHERS
### enable-weak-ssl-ciphers
Build support for SSL/TLS ciphers that are considered "weak"
Enabling this includes for example the RC4 based ciphersuites.
Spoiler: ZLIB
### zlib
Build with support for zlib compression/decompression.
Spoiler: ZLIB-DYNAMIC
### zlib-dynamic
Like the zlib option, but has OpenSSL load the zlib library dynamically
when needed.
This is only supported on systems where loading of shared libraries is supported.
Spoiler: 386
### 386
In 32-bit x86 builds, use the 80386 instruction set only in assembly modules
The default x86 code is more efficient, but requires at least an 486 processor.
Note: This doesn't affect compiler generated code, so this option needs to be
accompanied by a corresponding compiler-specific option.
Spoiler: NO-{PROTOCOL}
### no-{protocol}
no-{ssl|ssl3|tls|tls1|tls1_1|tls1_2|tls1_3|dtls|dtls1|dtls1_2}
Don't build support for negotiating the specified SSL/TLS protocol.
If `no-tls` is selected then all of `tls1`, `tls1_1`, `tls1_2` and `tls1_3`
are disabled.
Similarly `no-dtls` will disable `dtls1` and `dtls1_2`. The `no-ssl` option is
synonymous with `no-ssl3`. Note this only affects version negotiation.
OpenSSL will still provide the methods for applications to explicitly select
the individual protocol versions.
Spoiler: NO-{PROTOCOL}-METHOD
### no-{protocol}-method
no-{ssl|ssl3|tls|tls1|tls1_1|tls1_2|tls1_3|dtls|dtls1|dtls1_2}-method
Analogous to `no-{protocol}` but in addition do not build the methods for
applications to explicitly select individual protocol versions. Note that there
is no `no-tls1_3-method` option because there is no application method for
TLSv1.3.
Using individual protocol methods directly is deprecated. Applications should
use `TLS_method()` instead.
Spoiler: ENABLE-{ALGORITHM}
### enable-{algorithm}
enable-{md2|rc5}
Build with support for the specified algorithm.
Spoiler: NO-{ALGORITHM}
### no-{algorithm}
no-{aria|bf|blake2|camellia|cast|chacha|cmac|
des|dh|dsa|ecdh|ecdsa|idea|md4|mdc2|ocb|
poly1305|rc2|rc4|rmd160|scrypt|seed|
siphash|siv|sm2|sm3|sm4|whirlpool}
Build without support for the specified algorithm.
The `ripemd` algorithm is deprecated and if used is synonymous with `rmd160`.
Spoiler: COMPILER-SPECIFIC OPTIONS
### Compiler-specific options
-Dxxx, -Ixxx, -Wp, -lxxx, -Lxxx, -Wl, -rpath, -R, -framework, -static
These system specific options will be recognised and passed through to the
compiler to allow you to define preprocessor symbols, specify additional
libraries, library directories or other compiler options. It might be worth
noting that some compilers generate code specifically for processor the
compiler currently executes on. This is not necessarily what you might have
in mind, since it might be unsuitable for execution on other, typically older,
processor. Consult your compiler documentation.
Take note of the [Environment Variables](#environment-variables) documentation
below and how these flags interact with those variables.
-xxx, +xxx, /xxx
Additional options that are not otherwise recognised are passed through as
they are to the compiler as well. Unix-style options beginning with a
`-` or `+` and Windows-style options beginning with a `/` are recognized.
Again, consult your compiler documentation.
If the option contains arguments separated by spaces, then the URL-style
notation `%20` can be used for the space character in order to avoid having
to quote the option. For example, `-opt%20arg` gets expanded to `-opt arg`.
In fact, any ASCII character can be encoded as %xx using its hexadecimal
encoding.
Take note of the [Environment Variables](#environment-variables) documentation
below and how these flags interact with those variables.
Spoiler: ENVIRONMENT VARIABLES
### Environment Variables
VAR=value
Assign the given value to the environment variable `VAR` for `Configure`.
These work just like normal environment variable assignments, but are supported
on all platforms and are confined to the configuration scripts only.
These assignments override the corresponding value in the inherited environment,
if there is one.
Spoiler: MAKE VARIABLES
The following variables are used as "`make` variables" and can be used as an
alternative to giving preprocessor, compiler and linker options directly as
configuration. The following variables are supported:
AR The static library archiver.
ARFLAGS Flags for the static library archiver.
AS The assembler compiler.
ASFLAGS Flags for the assembler compiler.
CC The C compiler.
CFLAGS Flags for the C compiler.
CXX The C++ compiler.
CXXFLAGS Flags for the C++ compiler.
CPP The C/C++ preprocessor.
CPPFLAGS Flags for the C/C++ preprocessor.
CPPDEFINES List of CPP macro definitions, separated
by a platform specific character (':' or
space for Unix, ';' for Windows, ',' for
VMS). This can be used instead of using
-D (or what corresponds to that on your
compiler) in CPPFLAGS.
CPPINCLUDES List of CPP inclusion directories, separated
the same way as for CPPDEFINES. This can
be used instead of -I (or what corresponds
to that on your compiler) in CPPFLAGS.
HASHBANGPERL Perl invocation to be inserted after '#!'
in public perl scripts (only relevant on
Unix).
LD The program linker (not used on Unix, $(CC)
is used there).
LDFLAGS Flags for the shared library, DSO and
program linker.
LDLIBS Extra libraries to use when linking.
Takes the form of a space separated list
of library specifications on Unix and
Windows, and as a comma separated list of
libraries on VMS.
RANLIB The library archive indexer.
RC The Windows resource compiler.
RCFLAGS Flags for the Windows resource compiler.
RM The command to remove files and directories.
These cannot be mixed with compiling/linking flags given on the command line.
In other words, something like this isn't permitted.
$ ./Configure -DFOO CPPFLAGS=-DBAR -DCOOKIE
Spoiler: BACKWARD COMPATABILITY
Backward compatibility note:
To be compatible with older configuration scripts, the environment variables
are ignored if compiling/linking flags are given on the command line, except
for the following:
AR, CC, CXX, CROSS_COMPILE, HASHBANGPERL, PERL, RANLIB, RC, and WINDRES
For example, the following command will not see `-DBAR`:
$ CPPFLAGS=-DBAR ./Configure -DCOOKIE
However, the following will see both set variables:
$ CC=gcc CROSS_COMPILE=x86_64-w64-mingw32- ./Configure -DCOOKIE
If `CC` is set, it is advisable to also set `CXX` to ensure both the C and C++
compiler are in the same "family". This becomes relevant with
`enable-external-tests` and `enable-buildtest-c++`.
Spoiler: RECONFIGURE
### Reconfigure
reconf
reconfigure
Reconfigure from earlier data.
This fetches the previous command line options and environment from data
saved in `configdata.pm` and runs the configuration process again, using
these options and environment. Note: NO other option is permitted together
with `reconf`. Note: The original configuration saves away values for ALL
environment variables that were used, and if they weren't defined, they are
still saved away with information that they weren't originally defined.
This information takes precedence over environment variables that are
defined when reconfiguring.
Spoiler: DISPLAYING CONFIGURATION DATA
The configuration script itself will say very little, and finishes by
creating `configdata.pm`. This perl module can be loaded by other scripts
to find all the configuration data, and it can also be used as a script to
display all sorts of configuration data in a human readable form.
For more information, please do:
$ ./configdata.pm --help # Unix
or
$ perl configdata.pm --help # Windows and VMS
Spoiler: INSTALLATION STEPS IN DETAIL
Spoiler: CONFIGURE OPENSSL
Spoiler: AUTOMATIC CONFIGURATION
In previous version, the `config` script determined the platform type and
compiler and then called `Configure`. Starting with this release, they are
the same.
Spoiler: LINUX MAC
$ ./Configure [options...]
Spoiler: OPENVMS
$ perl Configure [options...]
Spoiler: WINDOWS
$ perl Configure [options...]
Spoiler: MANUAL CONFIGURATION
OpenSSL knows about a range of different operating system, hardware and
compiler combinations. To see the ones it knows about, run
$ ./Configure LIST # Unix
or
$ perl Configure LIST # All other platforms
Spoiler: UNIX
For the remainder of this text, the Unix form will be used in all examples.
Please use the appropriate form for your platform.
Pick a suitable name from the list that matches your system. For most
operating systems there is a choice between using cc or gcc.
When you have identified your system (and if necessary compiler) use this
name as the argument to `Configure`. For example, a `linux-elf` user would
run:
$ ./Configure linux-elf [options...]
Spoiler: CREATE YOUR OWN CONFIGURATION
If your system isn't listed, you will have to create a configuration
file named `Configurations/YOURFILENAME.conf` (replace `YOURFILENAME`
with a filename of your choosing) and add the correct
configuration for your system. See the available configs as examples
and read [Configurations/README.md](Configurations/README.md) and
[Configurations/README-design.md](Configurations/README-design.md)
for more information.
The generic configurations `cc` or `gcc` should usually work on 32 bit
Unix-like systems.
`Configure` creates a build file (`Makefile` on Unix, `makefile` on Windows
and `descrip.mms` on OpenVMS) from a suitable template in `Configurations/`,
and defines various macros in `include/openssl/configuration.h` (generated
from `include/openssl/configuration.h.in`.
Spoiler: OUT OF TREE BUILDS
OpenSSL can be configured to build in a build directory separate from the
source code directory. It's done by placing yourself in some other
directory and invoking the configuration commands from there.
Spoiler: UNIX
$ mkdir /var/tmp/openssl-build
$ cd /var/tmp/openssl-build
$ /PATH/TO/OPENSSL/SOURCE/Configure [options...]
Spoiler: OPENVMS
$ set default sys$login:
$ create/dir [.tmp.openssl-build]
$ set default [.tmp.openssl-build]
$ perl D:[PATH.TO.OPENSSL.SOURCE]Configure [options...]
Spoiler: WINDOWS
$ C:
$ mkdir \temp-openssl
$ cd \temp-openssl
$ perl d:\PATH\TO\OPENSSL\SOURCE\Configure [options...]
Paths can be relative just as well as absolute. `Configure` will do its best
to translate them to relative paths whenever possible.
Spoiler: BUILD OPENSSL
Build OpenSSL by running:
$ make # Unix
$ mms ! (or mmk) OpenVMS
$ nmake # Windows
This will build the OpenSSL libraries (`libcrypto.a` and `libssl.a` on
Unix, corresponding on other platforms) and the OpenSSL binary
(`openssl`). The libraries will be built in the top-level directory,
and the binary will be in the `apps/` subdirectory.
If the build fails, take a look at the [Build Failures](#build-failures)
subsection of the [Troubleshooting](#troubleshooting) section.
Spoiler: TEST OPENSSL
After a successful build, and before installing, the libraries should
be tested. Run:
$ make test # Unix
$ mms test ! OpenVMS
$ nmake test # Windows
**Warning:** you MUST run the tests from an unprivileged account (or disable
your privileges temporarily if your platform allows it).
See [test/README.md](test/README.md) for further details how run tests.
See [test/README-dev.md](test/README-dev.md) for guidelines on adding tests.
Spoiler: INSTALL OPENSSL
If everything tests ok, install OpenSSL with
$ make install # Unix
$ mms install ! OpenVMS
$ nmake install # Windows
Note that in order to perform the install step above you need to have
appropriate permissions to write to the installation directory.
The above commands will install all the software components in this
directory tree under `<PREFIX>` (the directory given with `--prefix` or
its default):
Spoiler: UNIX LINUX MAC
bin/ Contains the openssl binary and a few other
utility scripts.
include/openssl
Contains the header files needed if you want
to build your own programs that use libcrypto
or libssl.
lib Contains the OpenSSL library files.
lib/engines Contains the OpenSSL dynamically loadable engines.
share/man/man1 Contains the OpenSSL command line man-pages.
share/man/man3 Contains the OpenSSL library calls man-pages.
share/man/man5 Contains the OpenSSL configuration format man-pages.
share/man/man7 Contains the OpenSSL other misc man-pages.
share/doc/openssl/html/man1
share/doc/openssl/html/man3
share/doc/openssl/html/man5
share/doc/openssl/html/man7
Contains the HTML rendition of the man-pages.
Spoiler: OPENVMS
'arch' is replaced with the architecture name, `ALPHA` or `IA64`,
'sover' is replaced with the shared library version (`0101` for 1.1), and
'pz' is replaced with the pointer size OpenSSL was built with:
[.EXE.'arch'] Contains the openssl binary.
[.EXE] Contains a few utility scripts.
[.include.openssl]
Contains the header files needed if you want
to build your own programs that use libcrypto
or libssl.
[.LIB.'arch'] Contains the OpenSSL library files.
[.ENGINES'sover''pz'.'arch']
Contains the OpenSSL dynamically loadable engines.
[.SYS$STARTUP] Contains startup, login and shutdown scripts.
These define appropriate logical names and
command symbols.
[.SYSTEST] Contains the installation verification procedure.
[.HTML] Contains the HTML rendition of the manual pages.
Spoiler: ADDITIONAL DIRECTORIES
Additionally, install will add the following directories under
OPENSSLDIR (the directory given with `--openssldir` or its default)
for you convenience:
certs Initially empty, this is the default location
for certificate files.
private Initially empty, this is the default location
for private key files.
misc Various scripts.
The installation directory should be appropriately protected to ensure
unprivileged users cannot make changes to OpenSSL binaries or files, or
install engines. If you already have a pre-installed version of OpenSSL as
part of your Operating System it is recommended that you do not overwrite
the system version and instead install to somewhere else.
Package builders who want to configure the library for standard locations,
but have the package installed somewhere else so that it can easily be
packaged, can use
$ make DESTDIR=/tmp/package-root install # Unix
$ mms/macro="DESTDIR=TMP:[PACKAGE-ROOT]" install ! OpenVMS
The specified destination directory will be prepended to all installation
target paths.
Spoiler: COMPATABILITY ISSUES WITH PREVIOUS OPENSSL VERSIONS
Compiling Existing Applications
Starting with version 1.1.0, OpenSSL hides a number of structures that were
previously open. This includes all internal libssl structures and a number
of EVP types. Accessor functions have been added to allow controlled access
to the structures' data.
This means that some software needs to be rewritten to adapt to the new ways
of doing things. This often amounts to allocating an instance of a structure
explicitly where you could previously allocate them on the stack as automatic
variables, and using the provided accessor functions where you would previously
access a structure's field directly.
Some APIs have changed as well. However, older APIs have been preserved when
possible.
Spoiler: POST-INSTALLATION NOTES
With the default OpenSSL installation comes a FIPS provider module, which
needs some post-installation attention, without which it will not be usable.
This involves using the following command:
$ openssl fipsinstall
See the openssl-fipsinstall(1) manual for details and examples.
Spoiler: ADVANCED BUILD OPTIONS
Spoiler: ENVIRONMENT VARIABLES
Environment Variables
---------------------
A number of environment variables can be used to provide additional control
over the build process. Typically these should be defined prior to running
`Configure`. Not all environment variables are relevant to all platforms.
Spoiler: AR
AR
The name of the ar executable to use.
Spoiler: BUILDFILE
BUILDFILE
Use a different build file name than the platform default
("Makefile" on Unix-like platforms, "makefile" on native Windows,
"descrip.mms" on OpenVMS). This requires that there is a
corresponding build file template.
See [Configurations/README.md](Configurations/README.md)
for further information.
Spoiler: CC
CC
The compiler to use. Configure will attempt to pick a default
compiler for your platform but this choice can be overridden
using this variable. Set it to the compiler executable you wish
to use, e.g. gcc or clang.
Spoiler: CROSS_COMPILE
CROSS_COMPILE
This environment variable has the same meaning as for the
"--cross-compile-prefix" Configure flag described above. If both
are set then the Configure flag takes precedence.
Spoiler: HASHBANGPERL
HASHBANGPERL
The command string for the Perl executable to insert in the
#! line of perl scripts that will be publicly installed.
Default: /usr/bin/env perl
Note: the value of this variable is added to the same scripts
on all platforms, but it's only relevant on Unix-like platforms.
Spoiler: KERNEL_BITS
KERNEL_BITS
This can be the value `32` or `64` to specify the architecture
when it is not "obvious" to the configuration. It should generally
not be necessary to specify this environment variable.
Spoiler: NM
NM
The name of the nm executable to use.
Spoiler: OPENSSL_LOCAL_CONFIG_DIR
OPENSSL_LOCAL_CONFIG_DIR
OpenSSL comes with a database of information about how it
should be built on different platforms as well as build file
templates for those platforms. The database is comprised of
".conf" files in the Configurations directory. The build
file templates reside there as well as ".tmpl" files. See the
file [Configurations/README.md](Configurations/README.md)
for further information about the format of ".conf" files
as well as information on the ".tmpl" files.
In addition to the standard ".conf" and ".tmpl" files, it is
possible to create your own ".conf" and ".tmpl" files and
store them locally, outside the OpenSSL source tree.
This environment variable can be set to the directory where
these files are held and will be considered by Configure
before it looks in the standard directories.
Spoiler: PERL
PERL
The name of the Perl executable to use when building OpenSSL.
Only needed if builing should use a different Perl executable
than what is used to run the Configure script.
Spoiler: RANLIB
RANLIB
The name of the ranlib executable to use.
Spoiler: RC
RC
The name of the rc executable to use. The default will be as
defined for the target platform in the ".conf" file. If not
defined then "windres" will be used. The WINDRES environment
variable is synonymous to this. If both are defined then RC
takes precedence.
Spoiler: WINDRES
WINDRES
See RC.
Spoiler: MAKEFILE TARGETS
The `Configure` script generates a Makefile in a format relevant to the specific
platform. The Makefiles provide a number of targets that can be used. Not all
targets may be available on all platforms. Only the most common targets are
described here. Examine the Makefiles themselves for the full list.
Spoiler: ALL
all
The target to build all the software components and
documentation.
Spoiler: BUILD_SW
build_sw
Build all the software components.
THIS IS THE DEFAULT TARGET.
Spoiler: BUILD_DOCS
build_docs
Build all documentation components.
Spoiler: CLEAN
clean
Remove all build artefacts and return the directory to a "clean"
state.
Spoiler: DEPEND
depend
Rebuild the dependencies in the Makefiles. This is a legacy
option that no longer needs to be used since OpenSSL 1.1.0.
Spoiler: INSTALL
install
Install all OpenSSL components.
Spoiler: INSTALL_SW
install_sw
Only install the OpenSSL software components.
Spoiler: INSTALL_DOCS
install_docs
Only install the OpenSSL documentation components.
Spoiler: INSTALL_MAN_DOCS
install_man_docs
Only install the OpenSSL man pages (Unix only).
Spoiler: INSTALL_HTML_DOCS
install_html_docs
Only install the OpenSSL HTML documentation.
Spoiler: INSTALL_FIPS
install_fips
Install the FIPS provider module configuration file.
Spoiler: LIST-TESTS
list-tests
Prints a list of all the self test names.
Spoiler: TEST
test
Build and run the OpenSSL self tests.
Spoiler: UNINSTALL
uninstall
Uninstall all OpenSSL components.
Spoiler: RECONFIGURE
reconf
Re-run the configuration process, as exactly as the last time
as possible.
Spoiler: UPDATE
update
This is a developer option. If you are developing a patch for
OpenSSL you may need to use this if you want to update
automatically generated files; add new error codes or add new
(or change the visibility of) public API functions. (Unix only).
Spoiler: RUNNING SELECTED TESTS
You can specify a set of tests to be performed
using the `make` variable `TESTS`.
See the section [Running Selected Tests of
test/README.md](test/README.md#running-selected-tests).
Spoiler: TROUBLESHOOTING
Spoiler: CONFIGURATION PROBLEMS
### Selecting the correct target
The `./Configure` script tries hard to guess your operating system, but in some
cases it does not succeed. You will see a message like the following:
$ ./Configure
Operating system: x86-whatever-minix
This system (minix) is not supported. See file INSTALL.md for details.
Even if the automatic target selection by the `./Configure` script fails,
chances are that you still might find a suitable target in the `Configurations`
directory, which you can supply to the `./Configure` command,
possibly after some adjustment.
The `Configurations/` directory contains a lot of examples of such targets.
The main configuration file is [10-main.conf], which contains all targets that
are officially supported by the OpenSSL team. Other configuration files contain
targets contributed by other OpenSSL users. The list of targets can be found in
a Perl list `my %targets = ( ... )`.
my %targets = (
...
"target-name" => {
inherit_from => [ "base-target" ],
CC => "...",
cflags => add("..."),
asm_arch => '...',
perlasm_scheme => "...",
},
...
)
If you call `./Configure` without arguments, it will give you a list of all
known targets. Using `grep`, you can lookup the target definition in the
`Configurations/` directory. For example the `android-x86_64` can be found in
[Configurations/15-android.conf](Configurations/15-android.conf).
The directory contains two README files, which explain the general syntax and
design of the configuration files.
- [Configurations/README.md](Configurations/README.md)
- [Configurations/README-design.md](Configurations/README-design.md)
If you need further help, try to search the [openssl-users] mailing list
or the [GitHub Issues] for existing solutions. If you don't find anything,
you can [raise an issue] to ask a question yourself.
More about our support resources can be found in the [SUPPORT] file.
Spoiler: CONFIGURATION ERRORS
If the `./Configure` or `./Configure` command fails with an error message,
read the error message carefully and try to figure out whether you made
a mistake (e.g., by providing a wrong option), or whether the script is
working incorrectly. If you think you encountered a bug, please
[raise an issue] on GitHub to file a bug report.
Along with a short description of the bug, please provide the complete
configure command line and the relevant output including the error message.
Note: To make the output readable, please add a 'code fence' (three backquotes
` ``` ` on a separate line) before and after your output:
```
./Configure [your arguments...]
[output...]
```
Spoiler: BUILD FAILURES
If the build fails, look carefully at the output. Try to locate and understand
the error message. It might be that the compiler is already telling you
exactly what you need to do to fix your problem.
There may be reasons for the failure that aren't problems in OpenSSL itself,
for example if the compiler reports missing standard or third party headers.
If the build succeeded previously, but fails after a source or configuration
change, it might be helpful to clean the build tree before attempting another
build. Use this command:
$ make clean # Unix
$ mms clean ! (or mmk) OpenVMS
$ nmake clean # Windows
Assembler error messages can sometimes be sidestepped by using the `no-asm`
configuration option. See also [notes](#notes-on-assembler-modules-compilation).
Compiling parts of OpenSSL with gcc and others with the system compiler will
result in unresolved symbols on some systems.
If you are still having problems, try to search the [openssl-users] mailing
list or the [GitHub Issues] for existing solutions. If you think you
encountered an OpenSSL bug, please [raise an issue] to file a bug report.
Please take the time to review the existing issues first; maybe the bug was
already reported or has already been fixed.
Spoiler: TEST FAILURES
If some tests fail, look at the output. There may be reasons for the failure
that isn't a problem in OpenSSL itself (like an OS malfunction or a Perl issue).
You may want increased verbosity, that can be accomplished as described in
section [Test Failures of test/README.md](test/README.md#test-failures).
You may also want to selectively specify which test(s) to perform. This can be
done using the `make` variable `TESTS` as described in section [Running
Selected Tests of test/README.md](test/README.md#running-selected-tests).
If you find a problem with OpenSSL itself, try removing any
compiler optimization flags from the `CFLAGS` line in the Makefile and
run `make clean; make` or corresponding.
To report a bug please open an issue on GitHub, at
<https://github.com/openssl/openssl/issues>.
Spoiler: NOTES
Spoiler: MULTITHREADING
For some systems, the OpenSSL `Configure` script knows what compiler options
are needed to generate a library that is suitable for multi-threaded
applications. On these systems, support for multi-threading is enabled
by default; use the `no-threads` option to disable (this should never be
necessary).
On other systems, to enable support for multi-threading, you will have
to specify at least two options: `threads`, and a system-dependent option.
(The latter is `-D_REENTRANT` on various systems.) The default in this
case, obviously, is not to include support for multi-threading (but
you can still use `no-threads` to suppress an annoying warning message
from the `Configure` script.)
OpenSSL provides built-in support for two threading models: pthreads (found on
most UNIX/Linux systems), and Windows threads. No other threading models are
supported. If your platform does not provide pthreads or Windows threads then
you should use `Configure` with the `no-threads` option.
For pthreads, all locks are non-recursive. In addition, in a debug build,
the mutex attribute `PTHREAD_MUTEX_ERRORCHECK` is used. If this is not
available on your platform, you might have to add
`-DOPENSSL_NO_MUTEX_ERRORCHECK` to your `Configure` invocation.
(On Linux `PTHREAD_MUTEX_ERRORCHECK` is an enum value, so a built-in
ifdef test cannot be used.)
Spoiler: SHARED LIBRARIES
For most systems the OpenSSL `Configure` script knows what is needed to
build shared libraries for libcrypto and libssl. On these systems
the shared libraries will be created by default. This can be suppressed and
only static libraries created by using the `no-shared` option. On systems
where OpenSSL does not know how to build shared libraries the `no-shared`
option will be forced and only static libraries will be created.
Shared libraries are named a little differently on different platforms.
One way or another, they all have the major OpenSSL version number as
part of the file name, i.e. for OpenSSL 1.1.x, `1.1` is somehow part of
the name.
On most POSIX platforms, shared libraries are named `libcrypto.so.1.1`
and `libssl.so.1.1`.
on Cygwin, shared libraries are named `cygcrypto-1.1.dll` and `cygssl-1.1.dll`
with import libraries `libcrypto.dll.a` and `libssl.dll.a`.
On Windows build with MSVC or using MingW, shared libraries are named
`libcrypto-1_1.dll` and `libssl-1_1.dll` for 32-bit Windows,
`libcrypto-1_1-x64.dll` and `libssl-1_1-x64.dll` for 64-bit x86_64 Windows,
and `libcrypto-1_1-ia64.dll` and `libssl-1_1-ia64.dll` for IA64 Windows.
With MSVC, the import libraries are named `libcrypto.lib` and `libssl.lib`,
while with MingW, they are named `libcrypto.dll.a` and `libssl.dll.a`.
On VMS, shareable images (VMS speak for shared libraries) are named
`ossl$libcrypto0101_shr.exe` and `ossl$libssl0101_shr.exe`. However, when
OpenSSL is specifically built for 32-bit pointers, the shareable images
are named `ossl$libcrypto0101_shr32.exe` and `ossl$libssl0101_shr32.exe`
instead, and when built for 64-bit pointers, they are named
`ossl$libcrypto0101_shr64.exe` and `ossl$libssl0101_shr64.exe`.
Spoiler: RANDOM NUMBER GENERATION
Availability of cryptographically secure random numbers is required for
secret key generation. OpenSSL provides several options to seed the
internal CSPRNG. If not properly seeded, the internal CSPRNG will refuse
to deliver random bytes and a "PRNG not seeded error" will occur.
The seeding method can be configured using the `--with-rand-seed` option,
which can be used to specify a comma separated list of seed methods.
However, in most cases OpenSSL will choose a suitable default method,
so it is not necessary to explicitly provide this option. Note also
that not all methods are available on all platforms. The FIPS provider will
silently ignore seed sources that were not validated.
I) On operating systems which provide a suitable randomness source (in
form of a system call or system device), OpenSSL will use the optimal
available method to seed the CSPRNG from the operating system's
randomness sources. This corresponds to the option `--with-rand-seed=os`.
II) On systems without such a suitable randomness source, automatic seeding
and reseeding is disabled (`--with-rand-seed=none`) and it may be necessary
to install additional support software to obtain a random seed and reseed
the CSPRNG manually. Please check out the manual pages for `RAND_add()`,
`RAND_bytes()`, `RAND_egd()`, and the FAQ for more information.
Spoiler: ASSEMBLER MODULES COMPILATION
Compilation of some code paths in assembler modules might depend on whether the
current assembler version supports certain ISA extensions or not. Code paths
that use the AES-NI, PCLMULQDQ, SSSE3, and SHA extensions are always assembled.
Apart from that, the minimum requirements for the assembler versions are shown
in the table below:
| ISA extension | GNU as | nasm | llvm |
|---------------|--------|--------|---------|
| AVX | 2.19 | 2.09 | 3.0 |
| AVX2 | 2.22 | 2.10 | 3.1 |
| ADCX/ADOX | 2.23 | 2.10 | 3.3 |
| AVX512 | 2.25 | 2.11.8 | 3.6 (*) |
| AVX512IFMA | 2.26 | 2.11.8 | 6.0 (*) |
| VAES | 2.30 | 2.13.3 | 6.0 (*) |
---
(*) Even though AVX512 support was implemented in llvm 3.6, prior to version 7.0
an explicit -march flag was apparently required to compile assembly modules. But
then the compiler generates processor-specific code, which in turn contradicts
the idea of performing dispatch at run-time, which is facilitated by the special
variable `OPENSSL_ia32cap`. For versions older than 7.0, it is possible to work
around the problem by forcing the build procedure to use the following script:
#!/bin/sh
exec clang -no-integrated-as "[email protected]"
instead of the real clang. In which case it doesn't matter what clang version
is used, as it is the version of the GNU assembler that will be checked.
Spoiler: LINKS
[openssl-users]:
<https://mta.openssl.org/mailman/listinfo/openssl-users>
[SUPPORT]:
./SUPPORT.md
[GitHub Issues]:
<https://github.com/openssl/openssl/issues>
[raise an issue]:
<https://github.com/openssl/openssl/issues/new/choose>
[10-main.conf]:
Configurations/10-main.conf
Spoiler: STRAWBERRY PERL
Spoiler: DOWNLOAD
Strawberry Perl for Windows - Releases
strawberryperl.com
Spoiler: STRAWBERRY PERL README
=== Strawberry Perl (64-bit) 5.32.1.1-64bit README ===
Spoiler: WHAT IS STRAWBERRY PERL
* 'Perl' is a programming language suitable for writing simple scripts as well
as complex applications. See http://perldoc.perl.org/perlintro.html
* 'Strawberry Perl' is a perl environment for Microsoft Windows containing all
you need to run and develop perl applications. It is designed to be as close
as possible to perl environment on UNIX systems. See http://strawberryperl.com/
* If you are completely new to perl consider visiting http://learn.perl.org/
Spoiler: INSTALLATION INSTRUCTIONS
* If installing this version from a .zip file, you MUST extract it to a
directory that does not have spaces in it - e.g. c:\myperl\
and then run some commands and manually set some environment variables:
c:\myperl\relocation.pl.bat ... this is REQUIRED!
c:\myperl\update_env.pl.bat ... this is OPTIONAL
You can specify " --nosystem" after update_env.pl.bat to install Strawberry
Perl's environment variables for the current user only.
* If having a fixed installation path does not suit you, try "Strawberry Perl
Portable Edition" from http://strawberryperl.com/releases.html
Spoiler: HOW TO USE STRAWBERRY PERL
* In the command prompt window you can:
1. run any perl script by launching
c:\> perl c:\path\to\script.pl
2. install additional perl modules (libraries) from http://www.cpan.org/ by
c:\> cpan Module::Name
3. run other tools included in Strawberry Perl like: perldoc, gcc, gmake ...
* You'll need a text editor to create perl scripts. One is NOT included with
Strawberry Perl. A few options are Padre (which can be installed by running
"cpan Padre" from the command prompt) and Notepad++ (which is downloadable at
http://notepad-plus-plus.org/ ) which both include syntax highlighting
for perl scripts. You can even use Notepad, if you wish.
Spoiler: TEXT TEMPLATE PERL MODULE
Text::Template - Expand template text with embedded Perl - metacpan.org
Expand template text with embedded Perl
metacpan.org
Spoiler: NASM
Spoiler: DOWNLOAD
https://www.nasm.us/pub/nasm/releasebuilds/2.15.05/win64/nasm-2.15.05-win64.zip
Spoiler: WHAT IS NASM
NASM
Spoiler: GNU MAKE
Spoiler: DOWNLOAD
https://sourceforge.net/projects/gnuwin32/files/make/3.81/make-3.81.exe/download?use_mirror=phoenixnap&download=
Spoiler: WHAT IS GNUMAKE
GNU make
GNU make
www.gnu.org
Spoiler: MINGGW
Spoiler: DOWNLOAD
https://sourceforge.net/projects/mi...ngw-w64-release/mingw-w64-v9.0.0.zip/download
Spoiler: WHAT IS MINGGW
MinGW-w64
GCC for Windows 64 & 32 bits
www.mingw-w64.org
OTHER TOOLS FOR REFERENCE
Spoiler: NOTEPAD++
Spoiler: DOWNLOAD
https://github.com/notepad-plus-plus/notepad-plus-plus/releases/download/v8.3.1/npp.8.3.1.portable.x64.zip
Spoiler: WHAT AM I
This is the tool I use to help me write scripts better than just using a plain text editor.
FOR MORE OPENSSL DOCUMENTATION~! XDA HAS 80,000 WORD LIMIT XD
/docs/man3.0/man1/index.html
www.openssl.org
Make All Of This Easier~!
How To Use Chocolatey
========================= ============================================ HOW TO USE CHOCOLATEY ============== ============= Hi Friends~! This amazing package manager changed my Windoz life
forum.xda-developers.com
Happy studies!
Spoiler: USER ERROR INDUCED FORUM BUGZ
I DON'T UNDERSTAND HOW TWO SPOILERS KEEP ADDING THEMSELVES AT THE END OF MY THREAD ARBITRARILY AFTER EDITTING~! WHAT A NEAT LITTLE BUG THAT IS PRODUCING ITSELF, AS THIS TREE GROWS, LARGER, AND, LARGER <3
NOW IT ADDED 5 HAHAHA!
EDIT----NOW I KNOW WHAT IS CAUSING IT, IS WHEN I AM HAND TYPING THE SPOILERS AND I MISS /SPOILER, IT ADDS AS MANY MISSED SPOILERS TO THE END AS WAS MISSED IN THE ORIGINAL SPOILER TREE CODE. COOL~!
Spoiler: FURTHER LEARNING
I am watching this video now to learn about generating private and public key pairs, like we do with cryto currency~! How fun and neat to finally start to understand the backbones of this Tech
GitHub - pyca/cryptography: cryptography is a package designed to expose cryptographic primitives and recipes to Python developers.
cryptography is a package designed to expose cryptographic primitives and recipes to Python developers. - GitHub - pyca/cryptography: cryptography is a package designed to expose cryptographic prim...
github.com
What are OpenSSL BIOs? How do they work? How are BIOs used in OpenSSL?
I need some general information about OpenSSL BIO. Some kind of introduction to it. What is OpenSSL BIO? What is its general idea? I know that it is some kind of API for input/output. But how is it
stackoverflow.com
I finally did it~! Took me forever to figure out such simple things but my openssl toolset I'm sharing with y'all is portable and works out the gate :> all open source but mingw requires you keep their sources if you are interested in redistributing~!
Spoiler: SUCESSIO
And now through all this reinventing of the wheel~! I know what Chocolatey Is and Is used for~! I will figure this out better now haha.
Installing Chocolatey
Chocolatey is software management automation for Windows that wraps installers, executables, zips, and scripts into compiled packages. Chocolatey integrates w/SCCM, Puppet, Chef, etc. Chocolatey is trusted by businesses to manage software deployments.
chocolatey.org
Chocolatey Software Docs | Getting Started
Introduction to Chocolatey
docs.chocolatey.org
Packages
Chocolatey is software management automation for Windows that wraps installers, executables, zips, and scripts into compiled packages. Chocolatey integrates w/SCCM, Puppet, Chef, etc. Chocolatey is trusted by businesses to manage software deployments.
community.chocolatey.org
This package has been Blocked By Content Filtering On Some Browsers Or Even ISPs~!
I would say it's because The Shadow Corporations want to keep the code slaves on their IDE's but I digress...~~!
This is Mostly a Joke~! Mostly... ;-]
oMG Chocolatey was SO MUCH EASIER. I finally created my certificate; I'm going to sign my driver~! And whether or not it fixes the problem we will see. Regardless this has been a fun exercise and I will post a video with a visual example on my future chocolatey thread, about what I learned about IDEs, "Programming Languages", package installation on windows, system environment variables, etc~! Hopefully I will explain it in a way that makes sense
Something you should be aware of when using open source software, is that, even though it's free and states that you can use it in your own, Developed, Software, you absolutely must read the licences, readmes, and documentation, for sharing the work!
A GNU case, evidenced, that asks of, software developers, on freeware, can be enforced in court~!
Do your due diligence and read what the software providers add in text form for us because they did a lot of work for us to bring us what they do
13:13 mark
is Notepad++ superior to sublime??
Kross13 said:
is Notepad++ superior to sublime??
Click to expand...
Click to collapse
I will always be honest, I am Learning, So I do Not know~! What I advise is to try them both,, and learn the differences or "nuances" between the systems, so we can tell each other why we feel a certain way toward a certain program~! I will try out sublime now as I remember hearing about it years and years ago, but I wasn't at a basic enough level to understand what all of this is for.
Thank you for sharing this program with us~!
Thanks for this information. I will try it soon.
I think Notepad++ is the best free tool to help coding.
This is my opinion.
This video explains in super detail more about certificates, in ways I didn't even think of, in relation to man in the middle attacks on everyday websites we use.
Spoiler: Video
.
Since I'm thinking about it, and the idea of decentralization comes up, this video is interesting in explaining our present day internets' root
Spoiler: vid
Hello there. I wanted to share a link to a thread that "chains together" with this Thread. It is relating to Android Verified Boot which is a means of cryptographically protecting images. I am still learning so I will add links on that thread with interesting things. Notably the repository is available, there is a literal college thesis investigative study 33 pages, some good talks to listen to in the background and other things. Anyway it's here if anyone is interested
Learning About AVB Android Verified Boot (Boot.img dtb.img, vbmeta.img, and the "staging blob")
Edit-- after studying a couple days I understand why no modification to the images would work, which is due to AVB. I have a lot more studying to do and I will explain better. This thread is currently a mess of notes from a noob picking a kind...
forum.xda-developers.com
Oh and this is a subtle callout if anyone knows anything about avb please share on that thread. I am absolutely fascinated with this now and whelp... rabbit holes.

Categories

Resources