SD5650T V110

HiSilicon SD5650T-V110 is the SoC used in the QIVICON Home Base 2.

Memory map and interrupts

Based on hi_map.h.

base	length	device	IRQ	description
0x00000000	0x200000	ROM	--	Boot ROM
0x10100000	0x001000	SCTL		system control
0x10102000	0x001000	DDRC		DDR RAM controller
0x10104000	0x001000	TIMER01	47	timers 0 and 1
0x10105000	0x001000	TIMER23	48	timers 2 and 3
0x10106000	0x000100	GPIO0	49	GPIO bank 0
0x10107000	0x000100	GPIO1	50	GPIO bank 1
0x10108000	0x000100	GPIO2	51	GPIO bank 2
0x10109000	0x000100	GPIO3	52	GPIO bank 3
0x1010a000	0x000100	GPIO4	53	GPIO bank 4
0x1010b000	0x000100	GPIO5	54	GPIO bank 5
0x1010c000	0x000100	GPIO6	55	GPIO bank 6
0x1010d000	0x000100	GPIO7	56	GPIO bank 7
0x1010e000	0x001000	UART0	45	UART 0 (32-bit 8250)
0x1010f000	0x001000	UART1	46	UART 1
0x10180000	0x002000	A9_PERI	--	Cortex-A9 peripherals:
0x10180000	0x000100	- SCU
0x10180100	0x000100	- GIC_CPU	--	Generic Interrupt Controller, GIC v1
0x10180200	0x000100	- GLOBAL_TIMER
0x10180600	0x000100	- PRI_TIMER_WDT
0x10181000	0x001000	- GIC_DIST	--	GIC distributor
0x10a00000	0x010000	PCIE0
0x10a10000	0x010000	PCIE1
0x10a20000	0x000100	SFC	40	Serial Flash Controller
0x10a30000	0x000100	NFC	41	NAND Flash Controller
0x10a40000	0x001000	EHCI	38	EHCI USB controller
0x10a50000	0x001000	OHCI	39	OHCI USB controller
0x10a50000	0x010000	XHCI	39	XHCI USB controller (same address?)
0x10a60000	0x010000	SD/MMC	57
0x10a70000	0x010000	ETH	120, 121	[Ethernet]
0x11300000	0x080000	SRAM		internal RAM (until 0x11380000)
0x14380000	0x001000	MDIO0		[MDIO] 0
0x14400000	0x001000	MDIO1		MDIO 1
0x14880000	0x001000	CRG		clock/reset generator
0x14900000	0x001000	IOMUX		[I/O multiplexer]
0x16800000	0x001000	L2CACHE		level 2 cache controller
0x18000000	64 MiB	SFC CS0
0x1c000000	64 MiB	SFC CS1
0x20000000		NAND	--	NAND Flash Controller access window
0x40000000	0xa00000	PCIE0 mem	--	PCIe 0 memory space
0x40a00000	0x600000	PCIE0 io	--	PCIe 0 port I/O space
0x58000000	0xa00000	PCIE1 mem	--	PCIe 1 memory space
0x58a00000	0x600000	PCIE1 io	--	PCIe 1 port I/O space
0x80000000		DRAM	--	external DDR3 RAM

Boot ROM

The bootrom is located at 0 in the address space. It's really just 16 KiB (0x4000 bytes) long, but it repeats to fill a 2 MiB window.

UART 1 is used as a debug port (at the usual 3.3V and 115200 baud). Hitting Ctrl-C early in boot spawns the bootrom shell:

-------------------
- VER5610 bootrom -
-------------------
-
>> hit <ctrl+c> to stop autoboot: 1 
bootrom > help
cmd     - usage          - help
------------------------------------------------------------
boot    - boot           - read info from flash and boot
go      - go <addr>      - jump to application at 'addr'
help    - help           - print description of all commands
loady   - loady <addr>   - load file from the serial line
md      - md <addr>      - memory display
mw      - mw <addr>      - memory write
reset   - reset          - system reboot
run     - run <addr>     - execute application at 'addr'

(With proper command line editing and history support, btw!)

After 30 seconds, a watchdog kicks in and resets the board. Therefore it is necessary to occasionally re-enter the bootrom shell during longer sessions.

Commands

• boot continues with the normal boot flow (loading code from flash, etc.)
• reset performs a watchdog reset
• md prints 64 bytes at once, in hex+ASCII format. Data can be reconstructed later
  • A suffix may be specified to indicate width: b for byte (8 bits), l for long (32 bits), w for word (16 bits); The default is 32 bits.
  • A length may be specified as the second argument; it is the number of bytes to print minus 16 (in other words, the first line of output isn't counted).
• mw writes a value to memory. The arguments are address, value, repetitions. A width suffix may be specified as with md
• loady accepts data over serial, using the YMODEM protocol, and places it at the address specified in the parameter, or the default address of 0x11346000
• run executes code at the given address and passes extra arguments as argc/argv
• go is very similar but first disables cache and MMU. Programs written using mw or loady will not run correctly when started with go.

Normal boot flow

----------------------------------
- Flash type .......... [  NAND  ]
- Boot mode ........... [ NONSEC ]
- Read page0 .......... [   OK   ]
- DDR ................. [   OK   ]
- bootloader .......... [   OK   ]
----------------------------------
-
>> startup bootloader...

After determining the flash type and security mode, the bootrom resets and initializes the appropriate flash controller. It then loads and runs the DDR init code at internal memory address 0x11346000 and finally loads the bootloader to DDR memory address 0x80010000, disables cache/MMU and runs it.

Locations and sizes of the DDR and bootloader images are stored in a parameter struct in the first flash page, which is read with ECC mode 0.

Zero-page data structures

Page 0 is organized into several chunks of data, which start with a common header of two little-endian 32-bit words. The first is a type, the second is the size of the contained data (not including the 8-byte header).

Chunk 0x1002: Device parameters (short match)

This is a table of supported flash devices. Each entry consists of four bytes, and the table is terminated by an all-zero entry:

offset	type	description	NFC register bits
0	u8	Device ID	--
1	u8	ECC type	NFC_CON[9:11]
2	u8	page size shift	NFC_CON[1:3]
3	u8	unknown	NFC_BOOT_SET[0:1]

bootrom > md 0x20000000 0x2000
20000000: 00001002 00000038 000102f1 000102d1    ....8...........
20000010: 000102dc 000102a2 000102f2 000102a1    ................
20000020: 000102aa 000102da 000102ac 000102a3    ................
20000030: 000102d3 000102a5 000102d5 00000000    ................

The first (least-significant) byte is the Device ID, the second byte returned from the ONFI-defined flash command 90h.

For the Micron MT29F4G08ABDA (Device ID dc), the parameters are: ECC type = 2, page size shift = 1 (2048 bytes), bootset[0] = 0.

Chunk 0x1001: Device parameters (long match)

This chunk also contains a flash device table, but each entry is 12 bytes long:

offset	type	description
0	char[8]	full flash ID
8	u8	length of the ID
9	u8	ECC type
10	u8	page size shift
11	u8	unknown, corresponding to NFC_BOOT_SET[0]

20000040: 00001001 00000108 1590d198 00001476    ............v...
20000050: 00010206 a690dc2c 00000054 00010205    ....,...T.......
20000060: 3294d598 00005576 01030406 9590dc01    ...2vU..........
20000070: 00000056 00010205 9510dcad 00000054    V...........T...
20000080: 00010105 2594d5ad 00004144 00010106    .......%DA......
20000090: 9a94d5ad 00004274 00010106 9a94d7ad    ....tB..........
200000a0: 00004274 00010206 a690dc2c 00000054    tB......,...T...
200000b0: 00020308 4604482c 00000085 02020308    ....,H.F........
200000c0: 4a04482c 000000a5 02020308 2600382c    ,H.J........,8.&
200000d0: 00000085 01020308 4b04882c 000000a9    ........,..K....
200000e0: 02030408 4604682c 00000089 02020308    ....,h.F........
200000f0: 29d5d7ec 00004138 00010206 7284d5ec    ...)8A.........r
20000100: 00004250 00010106 72c5d7ec 00004254    PB.........rTB..
20000110: 00010206 7284d3ec 00004250 00010106    .......rPB......
20000120: b655d7ec 00000078 01010205 1d80f0c2    ..U.x...........
20000130: 00000000 00010104 9790dc03 00000057    ............W...
20000140: 00010205 00000000 00000000 00010200    ................

The flash page size is calculated as 1024 << shift.

For example, for the Micron MT29F4G08ABADA (ID 2c dc 90 95 56), the precise ID isn't listed in the table that I found on my machine. This is ok, because it's already covered by chunk 0x1002.

Chunk 0x2001: DDR init

This chunk contains the size of the DDR init code, for example (show below), 0x1000 (4096) bytes.

20000150: 00002001 00000028 00001000 00000000    . ..(...........
20000160: 00000000 00000000 00000000 00000000    ................
20000170: 00000000 00000000 00000000 00000000    ................

The DDR init code is found in the second page, located, for example, at offset 2048 (given a page size shift parameter of 1). Note that anything beyond page 0 is read with the ECC mode specified in the device parameters.

Chunk 0x2002: Bootloader

Similar to the previous one, this chunk contains the size of the bootloader, for example (show below), 0x1e800 bytes (122 KiB).

20000180: 00002002 00000028 0001e800 00000000    . ..(...........
20000190: 00000000 00000000 00000000 00000000    ................
200001a0: 00000000 00000000 00000000 00000000    ................

The bootloader is found directly after the DDR init code.

SMP boot

All of the above happens only for the primary CPU. Secondary CPUs (of which there is presumably one) are held in the bootrom until the register at address 0x1010011c has the value 0xc0ddffff. They may then jump to 0x80010000 (the bootloader's entry point in DRAM).

System control

offset	description
0x000	some clock source bits
0x008	boot flags:
0x008	x & 3: boot medium (ROM=0, SPI=1, NAND=2)
0x018	PCIE0_STAT0
0x038	PCIE0_STAT4
0x0a0	PERCTRL1
0x0a4
0x11c	SMP release signal
0x130	TESTREG1, relevant for SMP bringup
0x800	chip ID (variants: H=0x56100100, T=0x56102100)

Timers

The memory map for the four timers is listed in hi_map.h and used in hi_mach.c. Timer 1 and 3 are located 0x20 bytes after Timer 0 and 2, respectively.

offset	name	description
0x00	RELOAD
0x04	VALUE
0x08	CONTROL
0x0C	INTCLR
0x10	RIS	interrupt status?
0x14	MIS
0x18	BGLOAD

GPIO

The relevant symbols like sd5610x_gpio_bit_attr_set are not in the GPL release. Each bank (GPIO0 at 0x10106000, GPIO1 at 0x10107000, etc.) has the following layout, housing up to 32 GPIO lines:

offset	name	description
0x00	OUT	write bit to set output level
0x04	DIR	direction (0: input, 1: output)
0x08	FUNC	enable special function (0: GPIO, 1: function)
0x50	IN	read bit to get input level

I/O Multiplexer

The name IOMUX would imply that this block is configuring how peripheral controllers (SPI, etc.) are multiplexed onto pins of the chip.

Ethernet

The defconfig indicates that a Marvell Sky2 Ethernet chip is used, but this is a distraction. The SoC has a built-in Ethernet MAC and PHY.

The Ethernet MAC consists of eight channels (numbered 0-7), each of which correspond to one receive (RX) queue and one transmit (TX) queue. If interrupt reporting is enabled, events on channels 0-3 raise IRQ 120 at the SoC's Generic Interrupt Controller, events on channels 4-7 raise IRQ 121.

List of registers

offset	name	description
0x004		RX interrupt status (write 1 to clear)

MDIO

In addition to an Ethernet MAC, there is also an MDIO controller, to initiate communication with PHYs. The two instances of it are at 0x14380000 (conntected to internal PHY) and 0x14400000 (connected externally).

offset	name	description
0x04	CMD	initiate a transfer
0x0c	RDATA	value read from register
0x10	STATUS	status of current transfer

CMD

bits	name	description
0	GO	set to 1 to start a transfer
1	WRITE	transfer direction (0: read, 1: write)
2:6	PHY_ADDR	PHY address
7:11	REG_ADDR	register address
12:27	DATA	value that shoule be written to register, if WRITE

STATUS

bits	name	description
0:2	BUSY	non-zero while a transfer is running
3:4	COMPLETE	non-zero if a transfer completed successfully

USB

The EHCI/OHCI base addresses are known, so the problem seems simple at first, but there are some extra bits, belonging to the clock/reset generator. Also, an XHCI is listed at the same address as the OHCI.

NAND Flash Controller

The NAND flash controller's MMIO interface is at address 0x10a30000. Flash content can be accessed through a mapping at address 0x20000000. The register layout is listed in hi_nand_drv.h.

Clock/Reset Generator

The CRG can be found at MMIO address 0x14880000 (the resemblence to a racist dogwhistle is hopefully accidental). Registers are listed in hi_wdg.h, hi_crg_reg_s. Uses within the GPL source dump are rare, and spread out between hi_common.c, hi_mach.c, hi_pcie.c, and hi_wdg.c; search for HI_REG_BASE_CRG and g_pst_crg_reg.

List of registers

offset	name	desc.(zh), from GPL src.	description (en)
0x000	SC_SYSSTAT	软复位控制寄存器	soft reset control
0x004	ECSPLLL_CTRL0	ECS PLL控制寄存器	ECS PLL control
0x008	ECSPLLL_CTRL1	ECS PLL频率控制寄存器	ECS PLL frequency control
0x00c	ETHPLLL_CTRL0	ETH PLL控制寄存器	[Ethernet] PLL control
0x010	ETHPLLL_CTRL1	ETH PLL频率控制寄存器	Ethernet frequency control
0x014	SC_PEREN0	外设时钟使能寄存器0	peripheral clock enable 0
0x018	SC_PERDIS0	外设时钟禁止寄存器0	peripheral clock disable 0
0x01c	SC_PERCLKST0	外设时钟使能状态寄存器	peripheral clock enable status 0
0x020	SC_PEREN1	外设时钟使能寄存器1	peripheral clock enable 0
0x024	SC_PERDIS1	外设时钟禁止寄存器1	peripheral clock disable 1
0x028	SC_PERCLKST1	外设时钟使能状态寄存器1	peripheral clock enable status 1
0x02c	SC_RST_CTRL0	外设复位控制寄存器0	reset control 0, (active low)
0x030	SC_RST_CTRL1	外设复位控制寄存器1	" 1
0x034	SC_RST_CTRL2	外设复位控制寄存器2	" 2
0x038	CPU_CFG	CPU控制寄存器	CPU control
0x03c	HW_CFG	HW控制寄存器	HW(?) control
0x040	SC_PERCTRL0	外设控制寄存器0	peripheral control 0
0x044	SC_PERCTRL1	～1	" 1
0x048	SC_PERCTRL2	～2	" 2, GEMAC
0x04c	reserved	保留	--
0x050	SC_PERCTRL3	外设控制寄存器3	peripheral control 3
0x054	SC_PERCTRL4	～4	" 4
0x058	SC_PERCTRL5	～5	" 5
0x05c	SC_PERCTRL6	～6	" 6
0x060	SC_PERCTRL7	～7	" 7
0x064	SC_PERCTRL8	～8 (硬件看门狗使能控制)	" 8, watchdog enable control
0x068	RAM_CTRL_BUS	RAM控制寄存器	RAM control
0x06c	WDG_INIT_CFG	看门狗初始化配置信号	watchdog reset ("feed") register
0x070	OSC_CFG	晶振配置信号	crystal oscillator config signal
0x074	reserved (7)	保留	--
0x090	CRG_STAT0	CRG状态寄存器0	CRG state 0
0x094	DYING_GASP_PRE_INT	DYING_GASP中断寄存器	DYING_GASP interrupt
0x098	DYING_GASP_INT_MASK	DYING_GASP中断掩码寄存器	DYING_GASP interrupt mask
0x09c	DYING_GASP_INT_INSERT	DYING_GASP中断插入寄存器	DYING_GASP interrupt insertion
0x0a0	reserved (24)	保留	--
0x100	SC_CHIP_ID	芯片ID寄存器	chip ID (same as in [SCTL])
0x104	CRG_REV0	ECO预留寄存器0	CRG revision / reserved

Notes:

• To initiate a soft reset, write the chip ID masked with 0xffffff00 to SC_SYSSTAT, followed by the same but bit-inverted. For example, that would be 0x56102100 and 0xa9efdeff.
• To enable the watchdog, write 0x4F4E5452 ('ONTR') followed by 0x12345610 to SC_PERCTRL8. To disable the watchdog, write 0xabcd5610 and 0xed574447 ('\xedWDG').
• To reset (or "feed") the watchdog, write 0x55AA5A5A and 0xAA55A5A5 to WDG_INIT_CFG.

Reset signals

Resets are controlled by the bits in SC_RST_CTRL0..2, numbered LSB-first, allowing for up to 96 resets. For example, reset 0 the LSB of SC_RST_CTRL0, reset 63 is the MSB of SC_RST_CTRL1. The reset bits are active low: To assert a reset, clear the corresponding bit, to release a reset (let the hardware run), set the corresponding bit.

List of reset signals

reset	peripheral
15	USB
16	USB
17	USB
18	USB
26	Ethernet PHY
27	Ethernet PHY
33	Ethernet MAC
36	Ethernet MAC
38	Ethernet MAC/RGMII
40	Ethernet MAC
42	Ethernet MAC
44	Ethernet MAC
48	Ethernet MII 0
49	Ethernet MII 1
50	Ethernet MII 2
51	Ethernet MII 3
52	NAND Flash Controller
54	Ethernet RGMII
59	Ethernet MII 4
68	Ethernet RGMII
69	Ethernet RGMII

Clocks

Peripherl clocks are controlled by two sets of registers of enable, disable and status registers, allowing for up to 64 clocks. Clocks are enabled by setting the corresponding bit in SC_PERENx and disabled by setting the corresponding bit in SC_PERDISx. The current status is indicated by the corresponding bit in SC_PERCLKSTx.

List of clock signals

clock	peripheral
16	USB
22	Ethernet MAC (RGMII)
23	Ethernet MAC
24	Ethernet MAC
25	Ethernet MAC
26	Ethernet MAC
37	NAND Flash Controller
42	Ethernet RGMII
54	USB
58	Ethernet MII 0
59	Ethernet MII 1
60	Ethernet MII 2
61	Ethernet MII 3
62	Ethernet MII 4

Firmware

Some firmware must be stored in flash in order for an SD56xx-based system to boot.

DDR init

The DDR memory controller initialization code is a short program (2.6 KiB) that doesn't do any function calls and culminates in a single mov pc, lr instruction.

Bootloader

The bootloader is significantly larger, and for this reason, compressed.

>> startup bootloader...

DRAM       : 512MB       SYS        : 0xc0c00000 
STACK DATA : 0xc0020000  STACK SVC  : 0xc0030000 
STACK FIQ  : 0xc0040000  STACK ABT  : 0xc0050000 
STACK UND  : 0xc0060000  STACK IRQ  : 0xc0070000 
Memory     : total 511.5MB
Memory     : start 0xc2000000 available 64MB
Memory     : code 177KB bss 95KB highmem 416MB 0xc6000000

hi # 

Preloader (aka. SPL)

Before decompressing anything, the "pre-bootloader" stage does a bunch of initialization. Notably, it also brings up the secondary CPU core(s) by writing 0xc0ddffff to 0x1010011c. It sets up virtual memory (in the 0xc0000000 range) before calling into the main stage.

The preloader relocates itself from 0x80010000 to 0xc0080000.

Main stage

The main stage is LZMA-decompressed and loaded at virtual address 0xc4000000.

A few impressions:

• Interestingly, there seems to be a web interface for firmware updates, which identifies the device as a DSL Router. I guess that's correct most of the time
• Based on strings alone, the bootloader looks a lot like U-Boot. It has a bootm command ("boot application image from memory"), a bootdelay variable, it says things like Unknown command '%s' - try 'help' without arguments for list of all known commands
• It hints at a JFFS2 root filesystem: args_nand=mem=492M console=null,115200 root=mtd:rootfs ro rootfstype=jffs2
• The bootloader finally contains an Ethernet driver (previous stages didn't). It has code to load the OS image via TFTP, but the Ethernet link isn't detected fast enough
• The list of commands has been stripped down to just three:
  • bootm <address> interprets the data found at the given address as a uImage
  • sec_test <address> checks the validity of the page 0 data structures at
  • sec_init performs some hardware initialization on the SD/MMC device (0x10a60000)

The bootloader can be patched in RAM to pass custom arguments to the Linux kernel.