TomatoUSB

Ray123's post about the E3000 60K NVRAM builds prompted me to post this.

I have just over 1k left on my RT-N16. Nothing can be deleted - in fact soon I will have to consider other options such as saving user db (static ARP, restriction lists) elsewhere. The problem is, many of my routers have to be maintained by desk staff and this is just more things for them to screw up because they don't understand it.

The obvious answer is increased NVRAM size. To my way of thinking, 32k is a ridiculously small amount of NVRAM and like Ray, I find it surprising that it is so small even in these newer routers. Cisco routers have commonly have 128k, I believe. I'm not a programmer, but I'm wondering, as I expect you are, if the cfe can be altered or the firmware somehow modified to allow use of more space. Linksys have obviously realized this is going to be a problem and they addressed it with the E series.

I wanted to try the E3000 software on the RT, but the possibility seems to be that I'd get 60k OK but then cfe might then write things in the middle of it and overwrite data.

What, if any, are the options?

(This thread was moved to here from a series of PMs at linksys.org)

I am very reluctant to modify the CFE. That's just too too risky.

However, there is this: the NVRAM occupies the last eraseblock on the flash, but is programmatically restricted to the last 32KB of that EB. On most of these routers, the EB size is 64KB, so a full 32kb is wasted.

AFAIK, at bootup the CFE will look at the NVRAM to check that it is valid. If it isn't, it will erase it and set it to hard-coded default values.

So we cannot muck with it too much—-it must appear valid to the CFE. It might be possible to tweak the nvram-writing code in the kernel to segment the NVRAM into 2 pieces, one that the cfe expects and one that cfe will not see.

I could do this, I've been doing kernel programming for 20 years. There are some operational difficulties, though. These would have to be hashed out.

We really need to take any discussion over to a public board, either at linksys or (preferably) tomato.org

Regards,
Ray

That's very interesting, if the unused 32k could be put into use without changing the cfe, it would make it far easier to simply reflash the router with new firmware as usual.

BTW - after I sent that message, I realized that I don't know if source is even available for the ASUS cfe.

Source is available for the Asus CFE in the Asus GPL (or rather I did find some in one!), it even compiled but I didn't dare test it. The compiled one was smaller than current, possibly due to better compiler or incorrect compiler flags, but also possible that something missed out!

This is the CFE test you have to get around - nvram init is a shared function between bootloader and firmware, the cfe will have used whatever was current when it was compiled, it appears to check that the NVRAM_MAGIC code is in the header and maybe a crc of the contents header plus NVRAM_SPACE (0x8000 = 32kB)

/* Initialize NVRAM access accordingly. In case of invalid NVRAM, load defaults */
    if (nvram_init((void *)sih) > 0)
        restore_defaults = 1;

I remember looking at this before - the same area of code checks the reset button BEFORE setting nvram configured clockspeed - to workaround that old Linksys bug. It also waits for the reset button to be released, with key bounce wait etc before carrying on, which is all right and proper!

I also don't recommend mucking with the CFE, but if you do you could use the option to use lzma compression and reduce the CFE to 128kB which would be more use to 4MB flash than the Asus of course. I believe the compressed version was also comppiled at same time, so maybe there was a reason Asus didn't use it. dd-wrt have compressed cfe's working for 2MB routers, I don't know if they are fresh compiles or hexedits of a Linksys/Buffalo router that does use them.

Note that Asus RT-N16 uses a flash erase block size of 128kB, so every nvram commit replaces 128kB of flash, what a waste! It may be possible for the Linux firmware to always leave a CFE recognisable nvram in the right place, but if the CFE ever writes anything (and some used to on boot) it would blank anything it didn't like.

That would be a reasonable risk to take. It just means that you'd need to keep an up-to-date backup of your configuration so you could restore it.

IIRC, when I was pouring over the detail spec sheet on the flash chip used in the 520GU, it has a special protection feature. The 1st part of it (1st 16KB IIRC) can only be written or erased if you do some extraordinary thing. So that section is protected against an update that goes bad. I would guess that the tftp bootloader is tucked away there. At any rate, I've always been able to recover from a bad flash that bricked my router. BTW, it seems that if you have tftp running on your PC when you first power-on the router, it starts doing the upgrade without even having to mash the reset button. Wish I had known that sooner!

The first step to use the unused NVRAM area is to figure out how differential the items that must be visible to the CFE (if any) from items that can be tucked away elsewhere.

As a future enhancement, maybe it could do an lze compress. IIRC the kernel driver reads everything in from the nvram and builds an in-memory database. It then works entire within the im-memory stuff until it writes it to nvram, when you do a COMMIT.

OTOH, maybe we could keep only certain items (that the CFE wants to see) as-is, and compress everything else. Hmmmmm. Must investigate.

How to control this? Going forward is easy, but dropping back will mean that a large part of your NVRAM items would be lost. That won't do. TeddyBear already yelled at me once when I made an "improvement" that made something incompatible. ;-)

Can't see any point in protecting only 16kB of flash - that would only be good enough for MAC addresses etc?, you would need to protect the whole of the CFE, its a mini OS with Ethernet comms etc. The Router flash chip is pre-programmed before soldering in in production - that's why they often have stickers on them with the version. Very expensive to individually program, but clearly some routers are tested/configured using a serial console, that's why they leave us the pins!

Well its going to be easy to hide this from the CFE, I just took a look at the nvram on my WRT54G-TM (which has serial console plus working JTAG), with an 8MB flash it also has a 128kB Flash erase block size and 128kB nvram mtd block.

You can get a binary copy of the block with

cd /var
/tmp/var # dd if=/dev/mtdblock4 of=mtd4.bin

Then pick up the file mtd4.bin using winscp (using wine under Linux…) and look at it in a hexeditor.

Looking at the file, it is all "1"s until the MAGIC which has ascii code "FLSH" at offset 0x18000 - yes that's 96KB of blanks leaving exactly 32kB for the nvram vars including the header. There's no MAGIC or checksum at the end of the set nvram vars, just 4 extra nulls. All vars simple null terminated null strings.

There must be some history behind this strange and inefficient nvram storage, only reason for keeping size down that I can see is that it minimizes the size of memory needed to hold the local copy.

The AR7 ADSL routers I also hack have a 64kB config flash partition, of which 10kB is reserved for bootloader env vars, the rest is an XML document, binary compressed. This still causes grief however the 10kB env vars are in a form of filesystem with deleted/overwritten entries just marked and hidden until the kernel decides to defragment, and in the case of many production routers with "adam2" bootloaders there's a kernel bug (error by 1) which corrupts all the entries…

I have a WL-520GU. 4MB flash, 16MB ram. flash EB size is 64KB. No serial port.

BUT—YEA!!!! The TTL-to-USB serial cable I ordered (for $1.99 including shipping from Hong Kong to the US) just arrived today. Now I can open the case and do some soldering and I'll be able to get console access. If I don't fry anything.

The pmon is 128KB. It appears to be compressed, but I did a "strings" on it and see what is obviously NVRAM parameter strings. Including one that says "secret_code=56604474". Interesting.

I know why they did this "inefficient" simplistic way of nvram parameters. It's all that they needed. It's only when people like us decided to put all sorts of extre goodies into the routers that space gets tight in the NVRAM area. And I'm going dizzy right now, trying to trace what's going on in the CFE source code that I have. (Which isn't for my router at all.)

I agree that the CFE is better left alone - not everybody will be willing to risk updating it, and restoring the router with the dead bootloader is a way harder than with the CFE running… Ray seems to be on the right track though with modifying the nvram driver to split the nvram space into 2 separate pieces.

That would be relatively easy if we only had to care about keeping the nvram header in the right place… However the CFE also needs certain keys to be there - and if they are missing, it will create them and commit changes… Or worse - if it reads garbage from some of them ("clkfreq", for example) it may simply die. I suppose we can't just hardcode these few keys to decide what "area" of the nvram they should go to - the next CFE revision may introduce something new. Not really sure about the proper approach yet…

But since we got Ray interested and looking into it, it's a done deal ;). Well, almost - unless he gets too bored too soon, or too excited about fixing something else ;)…

@Ray - is that a Nokia DKU-5 equivalent cable? Watch-out I got one cheap one which didn't include any level shifting - ebay gave me my money back!

Is the "pmon" the CFE? It must include defaults to re-populate the nvram if that is blanked

My RT-N16 has secret_code=53939180

A comment in the CFE sourcefile giving nvram defaults (added to the CFE binary after compilation using the nvserial program) is that this is

1. The secret code for DDNS & WPS

The CFE source I have is not what was used to compile the CFE in my router - it appears to have been used most recently for an RT-N12

Sorry I am rather quiet, it's rather difficult to get good access to this site sometimes from here.

I used a copy of Nokia CA-42 USB data cable, but could only find drivers for XP. If you need any drivers take a look here first - I collected a lot of junk before finding something that worked for me. http://firmware.mooo.com/Flash%26Debrick/

Just a question - does JTAG (if supported) allow more flexibility with flashing than serial? (But JTAG software doesn't seem to be supporting RT-N16 yet).