GDBStub feature from ESP32

[MrMelon1232]

Hi,

An option that the ESP32 toolchain offers is the GDB debugging tool. In menuconfig of the build folders, it is possible to enable a feature called "Panic handler behaviour (invoke GDBStub)", so if a crash occurs, the terminal brings the user to a gdb environment for debugging. However, it seems that the freertos kernel from aws is missing a few things that allow this feature to work in comparison to esp-idf freertos folder under esp-idf/components/freertos/

This is the error output when trying to build the application:

freertos/vendors/espressif/esp-idf/components/esp32/gdbstub.c: In function 'getTaskInfo':
freertos/vendors/espressif/esp-idf/components/esp32/gdbstub.c:310:9: error: unknown type name 'TaskSnapshot_t'
  static TaskSnapshot_t tasks[STUB_TASKS_NUM];
freertos/vendors/espressif/esp-idf/components/esp32/gdbstub.c:314:15: error: implicit declaration of function 'uxTaskGetSnapshotAll' [-Werror=implicit-function-declaration]
   taskCount = uxTaskGetSnapshotAll(tasks, STUB_TASKS_NUM, &tcbSize);
               ^
freertos/vendors/espressif/esp-idf/components/esp32/gdbstub.c:317:46: error: request for member 'pxTCB' in something not a structure or union
   TaskHandle_t h = (TaskHandle_t)tasks[index].pxTCB;

In the freertos folder from esp-idf under esp-idf/components/freertos/, the structure TaskSnapshot_t is defined in
esp/esp-idf/components/freertos/include/freertos/task.h and looks like this:

/**
 * Used with the uxTaskGetSnapshotAll() function to save memory snapshot of each task in the system.
 * We need this struct because TCB_t is defined (hidden) in tasks.c.
 */
typedef struct xTASK_SNAPSHOT
{
	void        *pxTCB;         /*!< Address of task control block. */
	StackType_t *pxTopOfStack;  /*!< Points to the location of the last item placed on the tasks stack. */
	StackType_t *pxEndOfStack;  /*!< Points to the end of the stack. pxTopOfStack < pxEndOfStack, stack grows hi2lo
									pxTopOfStack > pxEndOfStack, stack grows lo2hi*/
} TaskSnapshot_t;

and the function uxTaskGetSnapshotAll is defined in esp/esp-idf/components/freertos/include/freertos/task.h and
esp/esp-idf/components/freertos/tasks.c and looks like this:

UBaseType_t uxTaskGetSnapshotAll( TaskSnapshot_t * const pxTaskSnapshotArray, const UBaseType_t uxArraySize, UBaseType_t * const pxTcbSz )
	{
		UBaseType_t uxTask = 0, i = 0;


		*pxTcbSz = sizeof(TCB_t);
		/* Fill in an TaskStatus_t structure with information on each
		task in the Ready state. */
		i = configMAX_PRIORITIES;
		do
		{
			i--;
			prvTaskGetSnapshotsFromList( pxTaskSnapshotArray, &uxTask, uxArraySize, &( pxReadyTasksLists[ i ] ) );
		} while( i > ( UBaseType_t ) tskIDLE_PRIORITY ); /*lint !e961 MISRA exception as the casts are only redundant for some ports. */

		/* Fill in an TaskStatus_t structure with information on each
		task in the Blocked state. */
		prvTaskGetSnapshotsFromList( pxTaskSnapshotArray, &uxTask, uxArraySize, ( List_t * ) pxDelayedTaskList );
		prvTaskGetSnapshotsFromList( pxTaskSnapshotArray, &uxTask, uxArraySize, ( List_t * ) pxOverflowDelayedTaskList );
		for (i = 0; i < portNUM_PROCESSORS; i++) {
			if( uxTask >= uxArraySize )
				break;
			prvTaskGetSnapshotsFromList( pxTaskSnapshotArray, &uxTask, uxArraySize, &( xPendingReadyList[ i ] ) );
		}

		#if( INCLUDE_vTaskDelete == 1 )
		{
			prvTaskGetSnapshotsFromList( pxTaskSnapshotArray, &uxTask, uxArraySize, &xTasksWaitingTermination );
		}
		#endif

		#if ( INCLUDE_vTaskSuspend == 1 )
		{
			prvTaskGetSnapshotsFromList( pxTaskSnapshotArray, &uxTask, uxArraySize, &xSuspendedTaskList );
		}
		#endif
		return uxTask;
	}

with 2 helper functions:

static void prvTaskGetSnapshot( TaskSnapshot_t *pxTaskSnapshotArray, UBaseType_t *uxTask, TCB_t *pxTCB )
	{
		if (pxTCB == NULL) {
			return;
		}
		pxTaskSnapshotArray[ *uxTask ].pxTCB = pxTCB;
		pxTaskSnapshotArray[ *uxTask ].pxTopOfStack = (StackType_t *)pxTCB->pxTopOfStack;
		#if( portSTACK_GROWTH < 0 )
		{
			pxTaskSnapshotArray[ *uxTask ].pxEndOfStack = pxTCB->pxEndOfStack;
		}
		#else
		{
			pxTaskSnapshotArray[ *uxTask ].pxEndOfStack = pxTCB->pxStack;
		}
		#endif
		(*uxTask)++;
	}

	static void prvTaskGetSnapshotsFromList( TaskSnapshot_t *pxTaskSnapshotArray, UBaseType_t *uxTask, const UBaseType_t uxArraySize, List_t *pxList )
	{
		TCB_t *pxNextTCB, *pxFirstTCB;

		if( listCURRENT_LIST_LENGTH( pxList ) > ( UBaseType_t ) 0 )
		{
			listGET_OWNER_OF_NEXT_ENTRY( pxFirstTCB, pxList );
			do
			{
				if( *uxTask >= uxArraySize )
					break;

				listGET_OWNER_OF_NEXT_ENTRY( pxNextTCB, pxList );
				prvTaskGetSnapshot( pxTaskSnapshotArray, uxTask, pxNextTCB );
			} while( pxNextTCB != pxFirstTCB );
		}
		else
		{
			mtCOVERAGE_TEST_MARKER();
		}
	}

Thank you!

[yourslab]

Sure, we will include these missing sources into the aws projects for ESP32. To unblock yourself, have you tried to include these missing files yourself to see if the feature works for you as expected?

[MrMelon1232]

Hi @yourslab,

Thank you for the quick response, it is very much appreciated! I have indeed tried to include these lines of code myself in the existing files but did not manage to get it to work due to so many dependencies required (example: same variables being used differently, wrong variable type that is already being used for something else such as
PRIVILEGED_DATA static List_t xPendingReadyList;
which was defined in esp/esp-idf/components/freertos/tasks.c like this:
PRIVILEGED_DATA static List_t xPendingReadyList[ portNUM_PROCESSORS ];

Thank You

[yourslab]

Happy to help! Could this be an issue for esp-idf then? tasks.c is required to build FreeRTOS, but it looks like gdbstub.c cannot build correctly along with it.

[MrMelon1232]

well it is an issue specific to the esp32 boards but it is not an esp-idf issue because gdbstub.c builds perfectly normal with esp-idf alone without using aws freertos. Originally, tasks.c is in a folder under esp/esp-idf/components/freertos/tasks.c in esp-idf. However, with aws freertos, the freertos folder with tasks.c has been moved to the freertos/freertos_kernel. So they are essentially the same file but with a few differences. The one in aws is missing few things to allow gdbstub.c to build correctly.

[yourslab]

Would it be possible for you to update projects/espressif/esp32/make/aws_demos/.project?
You can change PARENT-1-BASE_DIR_ROOT/freertos_kernel/tasks.c to point to PARENT-1-BASE_DIR_ROOT/vendors/espressif/esp-idf/components/freertos/tasks.c instead. You may have to update submodules, so please verify that the esp-idf directory under amazon-freertos has that file.
The trouble is that the esp-idf port of tasks.c is not in sync with the latest version of the kernel and is specific to esp32 boards. The aws_demos project may not have all the features available for a specific board, but we would like to keep the project in sync with the latest version of the kernel.

[MrMelon1232]

Hi @yourslab ,

I updated all submodules couldn't find the file tasks.c under the folder /vendors/espressif/esp-idf/components/freertos. I even attempted to checkout at v4.2 esp-idf (even if we're not supposed to use this version) and couldn't find the file.

EDIT: Adding the freertos folder manually and trying to build does not work as well even after pointing to PARENT-1-BASE_DIR_ROOT/vendors/espressif/esp-idf/components/freertos/tasks.c

[lundinc2]

Hello @MrMelon1232,

While we investigate an approach to this functionality, it may be easiest for you to fork the FreeRTOS-Kernel repository, and apply Espressif's changes directly to the kernel, instead of modifying the build to bring in multiple FreeRTOS kernel's.

For some background, the IDF SDK used in Amazon-FreeRTOS is different to to the standard IDF. It has been modified to work with the contents of Amazon FreeRTOS, which receives it's kernel version directly from https://github.com/FreeRTOS/FreeRTOS-Kernel.

I am not sure what kind of technical challenges bringing this feature in would be, but due to where the kernel is defined in this repository, it seems contentious with this implementation, as the kernel is being pulled in from a separate location, and does not have the Espressif modifications.

I will reach out to Espressif, and see if they have any advice on how to best approach this.

Thanks,

Carl

[MrMelon1232]

Hi @lundinc2,

Thank you for taking this feature into consideration, your efforts are appreciated.

[shubhamkulkarni97]

Hi @MrMelon1232,

As @lundinc2 correctly mentioned, ESP-IDF maintains some patches on top of FreeRTOS kernel to support this feature.
To enable this feature, changes will be required in freertos_kernel which is used by all vendors in Amazon FreeRTOS.
It will also take some time to evaluate these changes.

However, OpenOCD+GDB setup provides more control over device for debugging as compared to GDBStub. Is there any specific use-case to enable GDBStub?

[MrMelon1232]

Hi @shubhamkulkarni97,

Well using GDBSTUB allows easy debugging without the use of JTAG and brings you to the point of program crash. However, you are correct that using OpenOCD+GDB is better and allows more features but as mentioned in post #2822, I was not able to get this JTAG OpenOCD+GDB feature to work.

[dachalco]

Hi @MrMelon1232

There is a solution for #2822. Thank you @shubhamkulkarni97.

Could you try out the solution to see whether it enables you to use the JTAG debugger with your custom project?

[leegeth]

@MrMelon1232

Did the solution in #2822 help you resolve the problems you are facing? Are you looking for any further help from us?

[MrMelon1232]

Hi @leegeth @dachalco, I have tried this solution as I was the one who made the post. However, this feature request is different than #2822.

[leegeth]

@MrMelon1232,

Thank you for clarifying the differences between the 2 posts. I was of the impression that the summary from this post was that you are unblocked with OpenOCD+GDB debugging. I apologize for not noticing the feature request aspect of this post. I will reach out to Espressif about this feature request.

[leegeth]

@MrMelon1232,

Thank you for your patience. As already mentioned by @lundinc2 in this reply and @shubhamkulkarni97 in here, this feature needs some additional patches from Espressif to the FreeRTOS Kernel. We recommend you to request for this feature in either ESP32 forums or Espressif section of the FreeRTOS forums for better attention by Espressif engineers.
Since this feature request cannot be completed by FreeRTOS team, we are going ahead and closing the ticket. However, if you have any questions or concerns about this, please feel free to open another ticket and link this one.