See below...

73,

Burns Fisher, WB1FJ

AMSAT(R) Engineering -- Flight Software

On Wed, Nov 16, 2022 at 12:40 PM Bob Stricklin via pacsat-dev <pacsat-dev@amsat.org> wrote:

Nice write up Burns.

Thanks!

It sounds like the RTOS is being used as more of a dedicated task system and not really an OS. One possible advantage to this may be the memory needs are better defined. You also can’t be lost in a faulty task or orphaned task.

We can have memory leaks due to coding issues but hopefully those will be identified.

This seems to be the standard way an RTOS works. As I say, they CALL it an OS, but it is certainly different from what most of us think that means. Regarding memory leaks, that is not really possible unless you actually allocate memory from the heap. Local variable are all allocated on the stack when you call a routine and externs or module-level (or static) variables are allocated at link time. Oh, and yes, you do not LOAD tasks. They are just all compiled together. You can start and stop tasks, but there is little sense in doing that, I think. It does not save any memory except a bit in the heap.

My interest for asking the question was to help build an understanding for how much memory we need on the board. The code size you are talking about is not too large.

I am not clear where the code will sit which is in non volatile memory, the start up code. Would this be placed in flash and transferred to MRAM on power up for execution. So two 170K blocks of memory would be needed.

If the plan is not writing flash in orbit then how would changes be made to the program when problems develop?

All code actually lives in the flash memory and is executed there. So at boot time, there is no code loading done at all. It's already there. We have NEVER seen code in flash memory get corrupted in the Fox satellites. The MRAM can only be used for data. I believe that is where we will be putting messages to store and forward. On other satellites, we used MRAM for things like:

1) Whole orbit telemetry data

2) Min/Max telemetry data

3) Writable non-volatile storage for such things as "what was the last mode we were in", reset count, etc.

We have never had the capability of changing code on orbit. For all these processors, the code has to be in flash to be executed, and we always felt we should not write flash after launch. We froze the code months before delivery of the satellite and started doing testing. Any problems that turned up, we carefully considered whether changing code or keeping the bug was more risky. But maybe not writing in flash is old news and no longer an issue with the 570?? Something to check into.

If we have a Power on reset event (POR) where will the processor jump for the first code in a Fox or Pacsat? The CPU can probably be configured to go where you want in hardware, on chip or off chip.

Yes, a reset or a power cycle does the same thing. There is an address that the processor goes to in flash. I think that is hardwired (at least it was in the STM) but you just put a jump there if you don't want the code to start there.

I think you have all this thought though and would like to know what was the process for this? I will eventually see this in the schematics for the earlier designs but I am still working through those.

It took a lot of time back in the early days of Fox to even begin to understand how these processors worked and what the heck this "RT OS" thing is. One of the first tasks I had was recommending whether to use an OS at all and if so to choose one. As an OS developer, of course I wanted an OS (and it makes writing the tasks a lot easier than manually jumping among functions) but I had to do a lot of studying RTOSes before I really understood what it was all about. After that there was a lot of playing around with sample programs to understand the processor, and reading about RTOSes. One option was Altos which was written and designed by a couple AMSAT members for their model rocket business (Altos Metrum). That might actually have been a good choice, but at the time it was not yet written and there was little documentation and no other users.

I'm not 100% sure what is available in the Fox SVN about all this. Certainly the specs for the drivers that I wrote are there.

I have been trying to get operational on the existing satellites to get an idea of how much memory is being taken up by the user base. I still have a lot to learn in what is available to us in telemetry or how we can track this.

Maybe I'm not sure what it is that you want to "track" from telemetry. Memory use does not change minute by minute (well stack size does of course--each task has a fixed max stack size). If you wanted to see how it changed from satellite to satellite, we could rebuild each one and look at the link maps (I think that is still possible--certainly the source code has branches for each one).

I'm trying to do performance tests, btw. I'll get back.

Bob

On Nov 16, 2022, at 10:13 AM, Burns Fisher (AMSAT) <wb1fj@fisher.cc> wrote:

Bob asked about the amount of memory used by my FreeRTOS port. To be clear, there are two types of memory in the TMS570 (and in the STM32 as used in Fox-1 for that matter): Flash memory, which is non-volatile memory that holds that program and some constants, and RAM which is where volatile data is stored. In the past we have never planned to write the flash memory in orbit since it uses a charge pump which (we are told) is often the first thing to go in a radiation environment. It also has a limited number of write cycles (a few hundred or thousand?). Flash memory has an address and can be operated directly by the processor and code is generally executed directly out of it. This is totally different from the MRAM we have talked about. MRAM has no write cycle limit and seems to have a very good record in radiation. However MRAM is only accessible as a peripheral--it has no address and code can't be executed directly out of it.

The TMS570LS0914PGE, which is what is used on the RT-IHU has a total of 1280 KBytes of flash memory. The highest address in flash memory used by my OS port is 0x28aab8, which translates to about 170KBytes, of which 162 is code. And btw, the clock speed on this processor is 160MHz (or slower).

This processor also has 192KBytes of RAM of which 79K are currently used. However, 64K of that is used as heap storage by the OS of which only 32K is currently in use by the OS, so the heap could be made smaller. See below about this OS.

I think it is worth saying something about this RTOS (real time operating system) since this was a total surprise to me when I first started learning about it to write software for Fox-1. For those of us accustomed to Linux or WIndows or MacOS (or in my case an OS developer), one would hardly call this an operating system. It has three main functions:

1) Creating tasks and switching the processor between them to give the impression that they are operating simultaneously. This switching can be done either asynchronously (that is based on a timer), or it can be set up so that a task is switched away from only when that task blocks (i.e. it makes a call to wait for something).

2) Timers. One can create timers which will call some code when it expires. This code can then be used to release a blocked task. In other words one of the things that a task can wait for is for the timer to trigger.

3) Mutexes (of various forms) and queues: These are really the same thing---a task can wait on a mutex or queue, and a timer or another task or interrupt routine can release it by putting something in the queue.

That's it. Notice that I did not say anything about address spaces. All tasks share the same address space. If you are used to programming in Linux, think of the tasks as more like threads in a single process.

I also did not mention I/O. It is up to us to create all the I/O functions and to ensure that a task waiting for an I/O to complete does not block the rest of the tasks (i.e. it can't just loop). I have done this for all the peripherals used by Fox and Golf.

I also did not mention memory management. This is not strictly true. FreeRTOS does have a heap that one can use to alloc and free from. The OS itself uses it to allocate task, queue, and timer data structures. However, I would recommend never using this ourselves for two reasons. 1) We generally don't create and then destroy data structures. It is better to allocate them statically. 2) Deallocating data tends to leave holes in the heap so it is easy to get to a point where there is enough memory for an allocation you might want, but you can't allocate it because it is splintered. This is especially true with the relatively small amount of memory in our processor.

I hope this is helpful and forgive me if I have spent a lot of time stating the obvious. I'm kind of basing a lot of this on my memory of what I did not know 8 or 10 years ago when I started with all of this!

73,

Burns Fisher, WB1FJ

AMSAT(R) Engineering -- Flight Software

On Tue, Nov 15, 2022 at 9:54 PM Burns Fisher (AMSAT) <wb1fj@fisher.cc> wrote:

I believe that the easiest way to do this is to do a git clone (on a Linux system anyway). Then run CCS and I think you can import a CCS project. CCS/Eclipse has git builtin so after that you should be able to pull, checkin, merge, etc etc all from the IDE. Although I do checkin the CCS/Eclipse properties files, it might not be perfect so you may have to tweak around a bit to get it to work. If you have trouble lets work on it so we can try to figure out how to get it so multiple people with CCS can work together.

73,

Burns Fisher, WB1FJ

AMSAT(R) Engineering -- Flight Software

On Tue, Nov 15, 2022 at 9:49 PM Jim McCullers <wa4cwi@bellsouth.net> wrote:

Thanks for the work Burns.

Any particular steps to import this into CCS?

Jim McCullers

WA4CWI

The "main" branch of the PacSatx repository now has code which uses FreeRTOS and runs on the LaunchPad. Rather than starting from scratch and doing a new OS port and adding some code to do a few things, I started from Golf-TEE and removed much of the stuff that relates to Golf-TEE. Even if you go back to the root of this repo, you will not find anything that is not ok to be fully open source.

I left stuff in there that might be useful to PacSat. For example, there are I2c, SPI and CAN drivers, as well as device support code for a number of devices, including MRAM, temp sensors, serial port etc.

You will find that there are four tasks started up in the OS: Console, CAN, Command, and Radio. Here is the general idea:

1) Console: This is the task that interacts with a user during debugging and testing phases. I've left in the entire typed command parsing but taken out a lot of commands. The serial line that is used for console is actually implemented by the N2Het. The Rx and Tx lines are on the outside row of the LaunchPad, pins 25 and 27. This is TTL levels at 38,400 baud. (Ha, don't ask which is tx and which is rx. I just reversed the lines till it worked). You need a ground, too naturally.

2) CAN: Because CAN messages can arrive asynchronously, the easiest way to implement them is with a separate task. It works together with canDriver.c to send and receive CAN messages and to dispatch the received messages. Often these received messages on Golf will be telemetry, so they are dispatched to a telemetry collection task. That is removed to simplify the PacSat code. I don't know if we will use CAN at all, but the basic code is there.

3) Radio: This task drives the AX5043 both for transmitting BPSK telemetry and receiving AFSK commands. You will also see that this task can be set up as either a receiver or a transmitter. On Golf, it depends which of the two TMS570 processors it is running on. We have to design how this works for pacsat.

The tasks that generate telemetry have been removed, so you will see a few dead-end calls that are supposed to fetch telemetry data to give to the 5043 on the transmit side. On the receive side, the AFSK data received is handed off to the command task.

4) Command: This task gets a message from the Radio task when there is command data available. Its job is to decode and validate the command and then to execute it. Note that I often say "encrypt" or "decrypt" for commands. In fact, the commands themselves are not encrypted. However, they are validated by the standard method of transmitting an encrypted digest of the command and then decrypting the digest and comparing it against the digest of the received command. (This is pretty much the same way that downloading Windows upgrades works). The encryption/decryption requires a key. There is a default key that will be used normally. When we are approaching flight time, we will generate a "real" key to match the "real" command software and load it into the MRAM where the software can read it, but no human can.

My proposed next plan, subject to whatever we as a group decide, is to determine how to attach the AX5043 Rasp Pi shield to the LaunchPad and try to ensure that there is code both to receive AFSK and to transmit BPSK. Eventually, we will need to change this to AX.25 or whatever we decide to use for our message protocol, but at least we will know how to hook up the 5043 electrically and have some code to do a bit of testing.

73,

Burns Fisher, WB1FJ

AMSAT(R) Engineering -- Flight Software

-----------------------------------------------
pacsat-dev mailing list -- pacsat-dev@amsat.org
View archives of this mailing list at https://mailman.amsat.org/hyperkitty/list/pacsat-dev@amsat.org
To unsubscribe send an email to pacsat-dev-leave@amsat.org
Manage all of your AMSAT-NA mailing list preferences at https://mailman.amsat.org

-----------------------------------------------
pacsat-dev mailing list -- pacsat-dev@amsat.org
View archives of this mailing list at https://mailman.amsat.org/hyperkitty/list/pacsat-dev@amsat.org
To unsubscribe send an email to pacsat-dev-leave@amsat.org
Manage all of your AMSAT-NA mailing list preferences at https://mailman.amsat.org

-----------------------------------------------
pacsat-dev mailing list -- pacsat-dev@amsat.org
View archives of this mailing list at https://mailman.amsat.org/hyperkitty/list/pacsat-dev@amsat.org
To unsubscribe send an email to pacsat-dev-leave@amsat.org
Manage all of your AMSAT-NA mailing list preferences at https://mailman.amsat.org