diff --git a/docs/BootProcess.md b/docs/BootProcess.md new file mode 100644 index 00000000..eb3378d7 --- /dev/null +++ b/docs/BootProcess.md @@ -0,0 +1,95 @@ +Summary +--------- + +The high-level boot process for ONL is fairly straight forward, but there is a lot detail. + +At high-level, there are three phases +1. uBoot phase +2. the ONL Loader phase +3. The final ONL operating system + + +Detailed Boot Process +-------------------------- + +1. uBoot is the first level boot loader: http://www.denx.de/wiki/U-Boot +2. uBoot reads the 'nos_bootcmd' environmental variable from flash and runs the contents + ('nos' is Network Operating System) +4. If $nos_boot_cmd returns, uBoot loads and runs ONIE (see below) to download the ONL installer and install the ONL loader + a) The factory default $nos_boot_cmd is to a trival command that returned immediately, e.g., 'echo' +5. In normal operation, i.e., after ONIE has been run, $nos_boot_cmd is set to load and run the ONL Loader +6. The ONL loader boots the Linux kernel (later, the "boot kernel") +7. The ONL loader decides which SWI to run based on the URL in the file /etc/SWI + URL=`cat /etc/SWI` +8. The ONL loader runs `/bin/boot $URL` +9. The ONL loader retrieves the SWI file + a) if the URL is remote (e.g., http://, ftp://, etc.), verify that there is a locally cached copy + of the SWI in /mnt/flash2 or if not, download it + b) if the URL is local, verify that the device is accessible + c) if the URL is a Zero Touch Networking (ZTN) URL, the execute the ZTN protocol to get the SWI (see below) +10. The ONL loader reads the 'rootfs' file out of the SWI and mounts it using overlayfs[1] (SWI contents described below) + + + + + +Partition Layout +------------------ + +Switches typically have two flash storage device: a smaller flash (e.g., +64MB flash) for booting and a larger, mass storage device (e.g., compact +flash, 2+GB). + + +Smaller Boot Flash: + +Partition 1: uBoot +Partition 2: environmental variables (e.g., $nos_boot_cmd) +Partition 3: ONIE +Partition 4+: Free space (unused) + +Mass Storage Device: + +Partition 1: ONL loader kernel -- the format of this partition varies depending on what formats uBoot supports on the specific platform +Partition 2: ONL Loader configuration files (mounts as "/mnt/flash" both during the loader and the main ONL phases) +Partition 3: ONL SWitch Images (SWIs) partition (mounts as "/mnt/flash2" both during the loader and the main ONL phases) + +ONL file system layout +----------------------- + + root@onl-powerpc:/bin# df + Filesystem 1K-blocks Used Available Use% Mounted on + rootfs 72040 176 71864 1% / + devtmpfs 1024 0 1024 0% /dev + none 72040 176 71864 1% / + tmpfs 48028 148 47880 1% /run + tmpfs 5120 0 5120 0% /run/lock + /dev/sda2 71177 7 71170 1% /mnt/flash + /dev/sda3 3791960 98172 3693788 3% /mnt/flash2 + tmpfs 96040 0 96040 0% /run/shm + + +SWI +-------- + +Zip file contains: + + $ unzip -l ONL-2.0.0_ONL-OS_2015-12-12.0252-ffce159_PPC.swi + Archive: ONL-2.0.0_ONL-OS_2015-12-12.0252-ffce159_PPC.swi + Length Date Time Name + --------- ---------- ----- ---- + 97968128 2015-12-15 20:20 rootfs-powerpc.sqsh + 1063 2015-12-15 20:20 manifest.json + --------- ------- + 97969191 2 files + +1. 'rootfs-$ARCH' : the root file system for the running ONL +2. 'manifest.json' : a list of supported platforms, version information about ONL and the architecture + + + + +Footnotes +----------- + +[1] : https://kernel.googlesource.com/pub/scm/linux/kernel/git/mszeredi/vfs/+/overlayfs.current/Documentation/filesystems/overlayfs.txt diff --git a/docs/Building.md b/docs/Building.md index a2e8038c..951dfee3 100644 --- a/docs/Building.md +++ b/docs/Building.md @@ -19,15 +19,30 @@ All of the testing is done with Debian, other Linux distributions may work, but Build ONL Summary ------------------------------------------------------------ +The easiest way to build is to use the autobuild script: + + #> git clone https://github.com/opencomputeproject/OpenNetworkLinux + #> tools/autobuild/build.sh + +This will build a Debian 7 based ONL from the master branch + +To build a Debian 8 based ONL simply run: + + #> tools/autobuild/build.sh -8 + +If you would like to build by hand you can do the following: + #> git clone https://github.com/opencomputeproject/OpenNetworkLinux #> cd OpenNetworkLinux #> make docker # enter the docker workspace - user@system:/path/to/OpenNetworkLinux$ source setup.env # import variables necessary to build OpenNetworkLinux - #> make onl-x86 onl-powerpc # make onl for $platform (currently x86 or powerpc) + #> make amd64 ppc # make onl for $platform (currently amd64 or powerpc) + The resulting ONIE installers are in -$ONL/builds/installer/$ARCH/all/onl-$VERSION-all.installer, +$ONL/RELEASE/$SUITE/$ARCH/ONL-2.*INSTALLER, i.e. +RELEASE/jessie/amd64/ONL-2.0.0_ONL-OS_2015-12-12.0252-ffce159_AMD64_INSTALLER and the SWI files (if you want them) are in -$ONL/builds/swi/$ARCH/all/onl-$VERSION-all.swi. +$ONL/RELEASE/$SUITE/$ARCH/ONL*.swi. i.e. +RELEASE/jessie/amd64/ONL-2.0.0_ONL-OS_2015-12-12.0252-ffce159_AMD64.swi @@ -89,10 +104,11 @@ The list of packages for a given SWI are in Build a software image (SWI) for all powerpc platforms: ------------------------------------------------------------ - #> cd $ONL/builds/swi/powerpc/all + #> cd $ONL/builds/powerpc/swi #> make + #> cd builds #> ls *.swi - onl-27f67f6-powerpc-all-2014.01.27.10.59.swi onl-27f67f6-powerpc-all.swi + ONL-2.0.0_ONL-OS_2015-12-12.0252-ffce159_PPC.swi #> Build an ONIE-compatible installer for all powerpc platforms. @@ -100,8 +116,9 @@ This will incorporate the SWI you just built or build it dynamically if not. This installer image can be served to ONIE on Quanta or Accton platforms: ------------------------------------------------------------ - #> cd $ONL/builds/installer/powerpc/all + #> cd $ONL/builds/powerpc/installer/legacy #> make - #> ls *.installer - latest.installer onl-27f67f6-powerpc-all.2014.01.27.11.05.installer + #> cd builds + #> ls *INSTALLER + ONL-2.0.0_ONL-OS_2015-12-12.0252-ffce159_PPC_INSTALLER #> diff --git a/docs/GettingStarted.md b/docs/GettingStarted.md new file mode 100644 index 00000000..a676b3bb --- /dev/null +++ b/docs/GettingStarted.md @@ -0,0 +1,178 @@ +Getting Started +------------------------------------------------ +To install and run ONL you need is an ONL Compatible switch (see +http://opennetlinux.org/hcl) and the ONL installer binary. Every +ONL compatible switch ships with the ONIE installer environment installed +which gives you a multitude of ways of getting ONL installed on your switch. + +We document the easiest ways here (manual install via console and NFS) +but the http://onie.org website contains a variety of installation +methods including via USB, over the network, and even via ssh. + +The resulting installation has a default account ("root") with a default +password ("onl"). The network interface is disabled by default so that +you can change the root password before the system comes up. + + +ONL Manual Install +------------------------------------------------ +1) Attach a serial terminal to the switch +2) Boot switch and hit return to go to ONIE''s interactive mode + 2a) You must wait until after uboot has finished loading; if you + accidentally interupt uboot first, just run `boot` to continue + booting into ONIE +3) Download the ONL installer from http://opennetlinux.org and run it by hand + +Expected Serial Console Output (from an QuantaMesh LB9, other switches ouput will vary): + + U-Boot 2010.12 (Oct 08 2013 - 17:11:37) + + CPU: 8541, Version: 1.1, (0x80720011) + Core: Unknown, Version: 2.0, (0x80200020) + Clock Configuration: + CPU0:825 MHz, + CCB:330 MHz, + DDR:165 MHz (330 MT/s data rate), LBC:41.250 MHz + CPM: 330 MHz + L1: D-cache 32 kB enabled + I-cache 32 kB enabled + I2C: ready + DRAM: Detected UDIMM TS128MSD64V3A + Detected UDIMM(s) + DDR: 1 GiB (DDR1, 64-bit, CL=2.5, ECC off) + FLASH: 64 MiB + L2: 256 KB enabled + + LB9 U-Boot + Product Name : LB9 + Model Name : QUANTA LB9 + Serial Number : QTFCA63280001 + Part Number : 1LB9BZZ0STQ + Label Revision Number : 1 + Hardware Version : 1.0 + Platform Version : 0xb901 + Release Date : 2013/7/5 + MAC Address : 08:9e:01:ce:bd:2d + Set ethaddr MAC address = 08:9e:01:ce:bd:2d + In: serial + Out: serial + Err: serial + Net: TSEC0: PHY is Broadcom BCM5461S (2060c1) + TSEC0 + IDE: Bus 0: OK + Device 0: Model: 4GB CompactFlash Card Firm: Ver6.04J Ser#: CDE207331D0100001484 + Type: Hard Disk + Capacity: 3811.9 MB = 3.7 GB (7806960 x 512) + Hit any key to stop autoboot: 0 + ## Error: "nos_bootcmd" not defined + Loading Open Network Install Environment ... + Platform: powerpc-quanta_lb9-r0 + Version : 1.5.2-20131008154633 + WARNING: adjusting available memory to 30000000 + ## Booting kernel from Legacy Image at 04000000 ... + Image Name: quanta_lb9-r0 + Image Type: PowerPC Linux Multi-File Image (gzip compressed) + Data Size: 3479390 Bytes = 3.3 MiB + Load Address: 00000000 + Entry Point: 00000000 + Contents: + Image 0: 2762740 Bytes = 2.6 MiB + Image 1: 707380 Bytes = 690.8 KiB + Image 2: 9254 Bytes = 9 KiB + Verifying Checksum ... OK + ## Loading init Ramdisk from multi component Legacy Image at 04000000 ... + ## Flattened Device Tree from multi component Image at 04000000 + Booting using the fdt at 0x434f378 + Uncompressing Multi-File Image ... OK + Loading Ramdisk to 2ff53000, end 2ffffb34 ... OK + Loading Device Tree to 03ffa000, end 03fff425 ... OK + Cannot reserve gpages without hugetlb enabled + setup_arch: bootmem + quanta_lb9_setup_arch() + arch: exit + + ONIE: Using DHCPv4 addr: eth0: 10.7.1.10 / 255.254.0.0 + discover: installer mode detected. Running installer. + + Please press Enter to activate this console. ONIE: Using DHCPv4 addr: eth0: 10.7.1.10 / 255.254.0.0 + ONIE: Starting ONIE Service Discovery + + To check the install status inspect /var/log/onie.log. + Try this: tail -f /var/log/onie.log + +Now press RETURN here to jump into ONIE''s manual installer mode. You should see: + + ** Installer Mode Enabled ** + + ONIE:/ # + +Then simply download the latest ONL installer for the appropriate +architecture (powerpc or amd64) from the website and run it. + + ONIE:/ # install_url http://opennetlinux.org/binaries/latest-amd64.installer + + ONIE:/ # install_url http://opennetlinux.org/binaries/latest-powerpc.installer + Connecting to opennetlinux.org (107.170.237.53:80) + Open Network Installer running under ONIE. + Installer Version: Open Network Linux e148b7a (powerpc.all,2014.05.21.18.57,e148b7a90131c07eb8d49f74316baf8f2aae92c6) + Detected platform: powerpc-quanta-lb9-r0 + Installing in standalone mode. + Unpacking Open Network Linux installer files... + onl.powerpc-as4600-54t.loader + onl.powerpc-as5600-52x.loader + ... + + + + +ONL NFS Root Directory +------------------------------------------------ + +Given that the default installation of ONL does not persist files across +reboots (this is intentional -- flash disks should not be written to +as often as spinning disks), it is sometimes useful to have a normally +writable, larger disk available for the switch. Enter the NFS root +directory which enables a switch to boot ONL from a remote NFS partition. +While it is possible to simply fetch the SWI file from an NFS server +(keeping the same non-persisted behavior), the much more useful feature +is to have the root file system NFS hosted. + +To enable NFS mounted root partition: + +1) Run the ONL installer normally (e.g., via the manual mode per above) so that the ONL + loader is installed. + +2) Edit /mnt/flash/boot-config, enable DHCP, and change the SWI variable to point to a URL of the form "nfs://$ip[:port]/path/to/directory/". For example, on my machine, this looks like: + + # cat /mnt/flash/boot-config + SWI=nfs://10.6.0.4/home/robs/export/ly2-1/ # trailing '/' is critical + NETAUTO=dhcp # optional, but likely what you want + NETDEV=ma1 # leave untouched + +3) On server $ip, in /path/to/directory, unzip a target .SWI file, e.g., + + # wget http://opennetlinux.org/binaries/latest.swi + # unzip latest.swi + +4) unsquash the compressed root file system as directory 'rootfs-$arch': + + # unsquashfs -d rootfs-$arch rootfs-$arch.sqsh # e.g., $arch = 'powerpc'h + +Now reboot your switch and it should boot automatically into the NFS root file system. +Note that the SWI structure is still maintained: + + robs@sbs3:~/export/ly2-1$ ls -l + total 109048 + -rw-r--r-- 1 robs __USERS__ 3382017 Nov 4 22:28 initrd-powerpc + -rwxr-xr-x 1 robs __USERS__ 6942960 Nov 4 22:28 kernel-85xx* + -rw-r--r-- 1 robs __USERS__ 101322752 Nov 4 22:28 rootfs-powerpc.sqsh + drwxrwxr-x 22 robs __USERS__ 4096 Jan 2 18:21 rootfs-powerpc/ + -rw-r--r-- 1 robs __USERS__ 100 Nov 4 22:29 version + +That is: +* 'kernel-85xx' is the kernel image +* 'initrd-powerpc' is the initial RAM disk image +* 'rootfs-powerpc' is the base of the root filesystem +* 'version' is a string that identifies this SWI + +Note: If NFS root squash is set on the server, you might get a permission error while booting. To fix this, you can set 'no_disable_squash' in /etc/exports. However, be aware of the security implications as root on a client machine will now have the same access privilege on the files as root on the NFS server. diff --git a/docs/GettingStartedWedge.md b/docs/GettingStartedWedge.md new file mode 100644 index 00000000..bf608275 --- /dev/null +++ b/docs/GettingStartedWedge.md @@ -0,0 +1,144 @@ +Getting Started With the Accton Wedge +------------------------------------------------ +To install and run ONL on the Accton Wedge you will need ONIE and +the ONL wedge installer binary. Relevant pointers can be found at +https://opennetlinux.org/wedge + +Once installed, ONL has a default account ("root") with a default password +("onl"). The network interface is disabled by default so that you can change +the root password before the system comes up. You will need to enable the +network interface before you can run the FBOSS agent. + +FBOSS is installed and set to run the configuration created for a OCP ONL +on Wedge Demo. This configuration sets up the first physical QSFP port of +the wedge as 4 10G ports (via a break out cable) and configures vlans and +ip addresses on them. + +ONIE Manual Install +------------------------------------------------ +If your Accton Wedge does not have ONIE installed, you will need to install +it before you can proceed. + +1) Download the ONIE rescue image from http://opennetlinux.org/binaries/accton-wedge/onie-recovery-x86_64-accton_wedge_16x-r0.iso + +2) Burn the image onto a USB (a USB with a minimum size of 256M is necessary) + +3) Insert the USB into the front USB port on the Wedge + +4) Attach a serial terminal to the wedge + +5) Power on the wedge and wait for the BMC to finish booting + +6) Log into the bmc with the login: root password: 0penBmc + +7) Attach to the wedge microserver using the command sol.sh + +8) If the microserver shows the BIOS status screen press F2 to go to the BIOS +configuration menu + +9) In cases where the wedge microserver has already booted into the default +linux installation, you will need to either reboot (if possible) or hit ctrl-x, exiting to the BMC and issue the "wedge_power reset" command to power-cycle the +microserver, run sol.sh again and hit F2 when the BIOS status screen appears + +10) One you are in the BIOS configuration, move to the boot screen and change +the boot mode from UEFI to Legacy + +11) In the boot device list, make sure that the USB is set to #1 + +12) Hit esc-0 to or F10 to save and exit the configuration + +13) The system will now boot from the ONIE rescue USB + +14) When the ONIE Grub window appears, choose "Embed ONIE" + +15) ONIE will be installed on the system and the system will automatically +reboot + +16) *IMPORTANT* Remove the USB from the system before proceeding to the ONL install + +ONL Manual Install +------------------------------------------------ +1) Attach a serial terminal to the wedge + +2) Boot switch and choose "ONIE: Rescue" to go to ONIE''s interactive mode + +3) From the ONIE# prompt run "install_url http://opennetlinux.org/binaries/latest-wedge.installer" + +4) Wait for the install to finish and the system to reboot + +5) One the onl login prompt appears login with the username root and the +password "onl" + +6) You can now change the root password + +7) Configure the ma1 interface either via dhcp (dhclient ma1) or manually + +8) From the command prompt you can start fboss by using the command +"service fboss_wedge_agent start" + +9) The first time you start the fboss_wedge_agent service it will download +the OpenNSL library from the Broadcom github account. + +10) Once the library is installed, fboss_wedge_agent will start, using the +default configuration located at /etc/fboss/ocp-demo.json + +11) You can confirm that the fboss_wedge_agent is running by issuing the +command "service fboss_wedge_agent status" + +12) If fboss is running, you should see "[ ok ] fboss_wedge_agent is running." + +13) You can now execute the fboss_route.py script and configure the system. + +Modifying The fboss_wedge_agent configuration +------------------------------------------------ +In the /etc/init.d/fboss_wedge_agent script, you will locate a section where +the configuration file "FBOSS_DAEMON_OPTIONS" is set: + +OCP_DEMO="-config /etc/fboss/ocp-demo.json" +SAMPLE1="-config /etc/fboss/sample1.json" +SAMPLE2="-config /etc/fboss/sample2.json" +SAMPLE3="-config /etc/fboss/sample3.json" +FBOSS_DAEMON_OPTIONS=$OCP_DEMO + +The sample configurations are documented here: https://github.com/facebook/fboss/tree/master/fboss/agent/configs + +To change the configuration, simply change the variable from $OCP_DEMO to $SAMPLE1/2/3 or add your own configuration and set the variable to call it. + + +Expected output (you can also see videos at https://opennetlinux.org/wedge) + +ONIE: + + ONIE: Using DHCPv4 addr: eth0: 10.7.1.10 / 255.254.0.0 + discover: installer mode detected. Running installer. + + Please press Enter to activate this console. ONIE: Using DHCPv4 addr: eth0: 10.7.1.10 / 255.254.0.0 + ONIE: Starting ONIE Service Discovery + + To check the install status inspect /var/log/onie.log. + Try this: tail -f /var/log/onie.log + +Now press RETURN here to jump into ONIE''s manual installer mode. You should see: + + ** Installer Mode Enabled ** + + ONIE:/ # + +Then simply download the latest ONL wedge installer from the website and run it. + + ONIE:/ # install_url http://opennetlinux.org/binaries/latest-wedge.installer + + Connecting to opennetlinux.org (107.170.237.53:80) + Open Network Installer running under ONIE. + Installer Version: Open Network Linux 019421c (amd64.all,2015.11.17.06.23,019421c116940d87d5a41100d1b8a64cbba50252) + Detected platform: x86-64-accton-wedge-16x-r0 + Installing in standalone mode. + Unpacking Open Network Linux installer files... + ... + + Connecting tty=ttyS1 with /sbin/pgetty + + Open Network Linux 019421c (amd64.all,2015.11.17.06.23,019421c116940d87d5a41100d1b8a64cbba50252) + + onl-wedge login: + diff --git a/docs/PersistWorkflow.md b/docs/PersistWorkflow.md new file mode 100644 index 00000000..8363b753 --- /dev/null +++ b/docs/PersistWorkflow.md @@ -0,0 +1,55 @@ +# Rationale + +By default, the ONL file system is NOT persistent, meaning that if you +reboot, your changes will dissapear (!!). While this may sound suboptimal +at first, it does have the incredibly nice property of ensuring that many +classes of configuration and other problems can go away with a reboot. +This is particularly nice when you have a switch that may be headless +(no permanently connected console cable or keyboard). + +ONL accomplishes this with OverlayFS +(https://www.kernel.org/doc/Documentation/filesystems/overlayfs.txt). +As described at http://opennetlinux.org/docs/bootprocess, the ONL +switch image (.SWI file) contains a read-only root file system image. +The default ONL root file system is then a copy-on-write (using overlayfs) +view into that file system image. + +It has the following properites: + +* Any file that is editted/removed/etc is transparently copied into a RAM disk via overlayfs +* Thus, any changes to files appear as you would expect, until a reboot +* Any file that is uneditted remains backed by the /mnt/flash2 file system, so you + do not need to have enough RAM to store the entire rootfs. This is important with + switches that do not have much RAM to begin with. + +That said, ONL does have a provision to persist explicitly marked files +across a reboot. This document shows how this works. + + +# Persisting Files + +Just run `/sbin/persist /path/to/file` to mark a file as 'persisted'. This +file will be saved to the /mnt/flash persistent storage device and automatically +put back into place on reboot. Once a file has been persisted, it will always +be persisted across reboots. If you really want to unpersist a file, manually remove it from +'/persist/rootfs/path/to/file'. + +# Under the covers + +Running `/sbin/persist file` makes a hardlink of that file, e.g., /foo/bar/baz, to +/persist/rootfs/foo/bar/baz. + +The `/etc/init.d/restorepersist` script runs on bootup and does a number of things: + +* Restores the previously saved cpio archive from /mnt/flash/persist/rootfs into both / and /persist/rootfs +* Sets up hard links between /persist/rootfs/foo/bar/baz and /foo/bar/baz +* Starts a `watchdir` process for changes in /persist/rootfs + +`watchdir` in turn uses the inotify(3) subsystem to, upon a change, run `/sbin/savepersist /persist/rootfs`. + +And so, any change to a persisted file is noticed by watchdir and saved +to /mnt/flash in a cpio archive automatically using /sbin/savepersist. + +# Limitations + +You cannot persist any file that is read/used before `/etc/rcS.d/S03restorepersist` is run, including this script itself. Also, it is NOT recommended for persisting logging files, e.g., /var/log/*. While technically this will work, you will likely quickly exceed the write cycle limit of the underlying flash memory. Better to use a syslog server. diff --git a/docs/PortingGuide.md b/docs/PortingGuide.md new file mode 100644 index 00000000..5d62a084 --- /dev/null +++ b/docs/PortingGuide.md @@ -0,0 +1,198 @@ +Overview +======== + +Open Network Linux has a number of internal APIs to simplify porting to +new hardware. + +To support a new switch/device, there are three large software components +that need device-specific drivers: + +1. The ONL/ONIE installer -- how to actually install and boot ONL using ONIE +2. The ONLP platform drivers -- how to manage hardware once ONL is running +3. Any packet forwarding device specific settings + (e.g., front panel port mappings, pre-emphesis settings) + +ONL provides plugable driver modules for (1) and (2) but currently +provides no support for (3) -- this is work in progress. + +The rest this PortingGuide is divided into two parts: (1) creating +the ONIE installer driver and (2) creating the ONLP platform driver. + +ONL Installer +================== + +About ONIE +---------- + +ONIE (the Open Network Installation Environment - http://onie.org) +is a small piece of software that ONL expects to exist on every +switch, pre-installed by the switch vendor. ONIE provides the +installation and management utilities to install/uninstall/rescue +a Network Operating System (a "NOS") like ONL. While ONIE is a +stand alone operating system in its own right, it is intentionally +stripped down and has few features outside of the bare minimum +needed to bootstrap a system and invoke an NOS installer program. + +ONL Installer Operation +----------------------- + +ONL has an ONIE-compatible installation script called the 'ONL +Installer'. That is, the ONL installer is a device-specific +self-extracting shell script that, when run from ONIE, partitions +available storage, installs the ONL switch image file (SWI) on to +disk, and sets up the underlying boot loader (e.g., uboot or grub) +with the appropriate settings so that on boot, by default, ONL loads +correctly on the device. So, to port ONL to a new switch, a number +of installer drivers, including for disk partitioning, installation, +booting, and python integration need to be written. + +ONL Installer Example Code +-------------------------- + +TL; DR :: If you want to just jump to the code, look at the drivers +in $ONL/packages/platforms/$vendor/$platform/platform-config/r(0|1)/* -- in particular, the +powerpc-as5710-54x-r0b driver is a good example. Copy the directory/file layout and naming conventions and +see how the driver code is called from $ONL/builds/powerpc/installer/legacy/builds/ppc-installer.sh + +ONL Installer file layout +------------------------- +All the installer drivers are stored in $ONL/packages/platforms/$vendor/$platform/platform-config, +where $platform corresponds to the ONIE platform identifier string and $vendor is the name of the platform +vendor e.g. accton, quanta, wnc. This string is used to identify which set of drivers to load (ONL supports +many systems) at boot time and is critical that it matches the ONIE identifier exactly. The rest of the +directory structure for the installer driver is as follows: + + ./$platform/$release/ + ./$platform/$release/Makefile # copy from existing driver + ./$platform/$release/src/lib/boot/detect.sh # Script that returns 0 if run on $platform, 1 otherwise + ./$platform/$release/src/lib/boot/$platform # Script run on boot that populates device/hardware + # specific ONL OS abstractions (see below) + ./$platform/$release/src/lib/install/$platform.sh # Script called from installer.sh to partition + # and install ONL and setup boot params (see below) + ./$platform/src/python/$plaform/__init__.py # Platform specific python library (see below) + +ONL Installer src/boot drivers +------------------------------ + +The $platform/src/lib/boot/$platform script is in charge of writing +the ONL boottime hardware abstraction configuration. Because each +device has a different storage driver, storage device, management +network device, etc., ONL uses a series of config files to map the +specific hardware (an ethernet driver) to a device-agnostic form +(e.g., primary management interface). + +The following files need to be populated by the $platform boot script: +* /etc/onl/net : The PCI device name and interface name of the management ethernet +* /etc/onl/mounts : The devices that correspond to /mnt/flash and /mnt/flash2 at boot +* /etc/fw_env.config : The device and configuration of the firmware device, for fw_printenv, fw_setenv + + +ONL Installer src/install drivers +--------------------------------- +The $platform/src/lib/install/$platform.sh driver is called from the main installer script and has the job +of: +1) Identifying and partitioning the device specific storage +2) Deciding the file format of the target boot partition to install the ONL loader image +3) Deciding how to update the bootloader, e.g., in the case of uboot, what the $nos_bootcmd variables + should be. + +The contents of this driver are sourced into the installer.sh and +so the variables and functions are called in various places. There +are an excess of ways that switches can be built (what storage +types, etc.) and how they boot, so seading the installer.sh code +along with the example platform driver code seems to be the best +bet here -- FIXME! + +For example, looking at the driver +`$ONL/packages/platforms/accton/powerpc-accton-as5710-54x/platform-config/r0b/src/lib/install/powerpc-as5710-54x-r0b.sh` +the driver first sets "platform_loader_raw=1" which tells the +installer that the boot partition has no file format (as opposed +to ext2fs or fat - it is type 'raw'). Then it has the platform_bootcmd as: + +`platform_bootcmd='usb start; usbboot 0x10000000 0:1; setenv bootargs console=$consoledev,$baudrate onl_platform=powerpc-as5710-54x-r0b; bootm 0x10000000'` + +Which is a string of uboot commands, that: + +1. usb start -- initialize the usb subsystems +2. Using the usb boot loader, load the contents of device 0, partition 1 (0:1) into memory range 0x10000000 +3. Pass a number of env variables to the kernel at boot via $bootargs +4. Actually just jump to memory 0x10000000 and start running (bootm 0x100000000) + +The sequence of exact boot commands will vary with version of uboot +(or other boot loader), available storage, and other device specific +properties. + + +ONL Installer src/python drivers +-------------------------------- + + +Open Network Linux Platform ("ONLP") APIs +========================================= + +Every new networking switch/router/box has a unique layout of which +devices (fans, power supplies, LEDs, SFP/SFP+/QSFP, temperature +sensors, etc.) connect to which I/O devices (I2C, GPIO, etc.) and +how they are managed (FPGA, CPLD). Rather than mandate one hardware +approach or assume that there exists a BIOS to take care of this +work for us (some platforms have a BIOS, some do not; some drivers +are not supported by BIOS), ONL has created an abstraction layer +to inventory, manage, and monitor these devices. + + +ONLP Application APIs +--------------------- + +If you want to create an application in ONL that builds on top of the +platform, the "application to platform" APIs are found in: + + $ONL/packages/base/any/onlp/src/onlp/module/inc/onlp + +This section will have to become better documented, but look at the example +code in the `onlpdump` driver for how to call the ONLP APIs as an application. + +At a high-level, each hardware device is given a unique Object ID +(OID). Each type of device has a number of different properties that +is querable/programmable (e.g., read/set fan speed, query an SFP+ port +status, etc.) and a mechanism for negotiating hardware capabilities +(e.g., is the fan speed setable? reversible? does this SFP have a +hardware interupt bit on link down?). + +The ONLP API is has internal locking so it supports multiple concurrent +callers. That said, there is no inter-application internal hardware +contention resolution, so, if for example one application wants the fans +at 25% and another wants them at 75%, the last caller wins. + +Applications start by getting a tree of OIDs from the platform using the + onlp_sys_init(void) and onlp_sys_info_get() calls. There exist a number +of macros for interogating OID types in oid.h, ala ONLP_OID_IS_*(). + + +ONLPI Driver APIs +----------------- + +If you want to create a driver so that your new hardware can work with +ONL, the "platform to hardware" APIs are found in: + + $ONL/packages/base/any/onlp/src/onlp/module/inc/onlp/platformi + +This section will have to become better documented, +but look at the example driver 'onlpie' implementation at +$ONL/packages/base/any/onlp/src/onlpie/module/src/. Many driver +implementations have been written and they will become available over +time. + +At a high-level, the driver is responsible for providing implementations +of the various 'platformi' APIs, e.g., sysi.h, fani.h, etc. Whether +these implementations are provided via user-space or kernel space is an +implementation detail left to the driver maintainer. + +In terms of programming paradigm, the application calls into the platform +code (see above) and then the platform code calls into the driver. The main +platform code handles locking to ensure that the underlying driver code does +not need to be re-entrant or handle concurrent API calls. This dramatically +simplifies the ONLPI driver code and we have found in most cases that code +from existing projects (e.g., from an ODM diagnostic utilities) can be readily +cut and pasted into place. + +Feedback on these APIs are welcome. diff --git a/docs/SupportedHardware.md b/docs/SupportedHardware.md new file mode 100644 index 00000000..9a4f9752 --- /dev/null +++ b/docs/SupportedHardware.md @@ -0,0 +1,62 @@ +Hardware Support +================ +Because of the HTML formatting, this page may be best viewed from + + + +Quanta +------ + + + + + + + + + + +
Device Ports CPU Forwarding Support Status
QuantaMesh T1048-LB9 48x1G + 4x10G FreeScale P2020 Broadcom BCM56534 (Firebolt3) Supported and Tested
QuantaMesh T1048-LB9A 48x1G + 4x10G FreeScale P2020 Broadcom BCM56534 (Firebolt3) Supported, no ONIE, not regularly tested
QuantaMesh T3048-LY2 48x10G + 4x40G FreeScale P2020 Broadcom BCM56846 (Trident+) Supported and Tested
QuantaMesh T3048-LY8 48x10G + 6x40G Intel Rangely C2758 x86 Broadcom BCM56854 (Trident2) Supported and run in the lab
QuantaMesh T5032-LY6 32x40G Intel Rangely C2758 x86 Broadcom BCM56850 (Trident2) Supported and run in the lab
+ + +Accton/Edge-Core +------ + + + + + + + + + + + + + + + +
Device Ports CPU Forwarding Support Status
Accton AS4600-54T 48x1G + 4x10G FreeScale P2020 Broadcom BCM56540 (Apollo2) Supported and Tested
Accton AS5600-52X 48x10G + 4x40G FreeScale P2020 Broadcom BCM56846 (Trident+) Supported and Tested
Accton AS5610-52X 48x10G + 4x40G FreeScale P2020 Broadcom BCM56846 (Trident+) Supported and Tested
Accton AS5710-54X 48x10G + 6x40G FreeScale P2041 Broadcom BCM56854 (Trident2) Supported and Tested
Accton AS6700-32X 32x40G FreeScale P2041 Broadcom BCM56850 (Trident2) Supported and Tested
Accton AS5712-54X 48x10G + 6x40G Intel Rangely C2538 x86 Broadcom BCM56854 (Trident2) Supported and Tested
Accton AS6712-32X 32x40G Intel Rangely C2538 x86 Broadcom BCM56850 (Trident2) Supported and Tested
Accton AS5812-54T 48x10G + 6x40G Intel Rangely C2538 x86 Broadcom BCM56864 (Trident2+) Supported
Accton AS5812-54X 48x10G + 6x40G Intel Rangely C2538 x86 Broadcom BCM56864 (Trident2+) Supported
Accton AS6812-32X 32x40G Intel Rangely C2538 x86 Broadcom BCM56864 (Trident2+) Supported
+ +DNI/Agema +--- + + + + + + +
Device Ports CPU Forwarding Support Status
AG-7448CU 48x10G + 4x40G FreeScale P2020 Broadcom BCM56845 (Trident) Supported and Tested
+ +Dell +--- + + + + + + + + + +
Device Ports CPU Forwarding Support Status
S4810-ON 48x10G + 4x40G FreeScale P2020 Broadcom BCM56845 (Trident) Supported and Tested
S4048-ON 48x10G + 6x40G Intel Atom C2338 Broadcom BCM56854 (Trident2) Supported and Tested
S6000-ON 32x40G Intel Atom S1220 Broadcom BCM56850 (Trident2) Supported and Tested