openamp / libmetal Goto Github PK

Hello,

I have a device with 2 blocks of memory-mapped register that appear as 2 IO regions within a single device.

I've opened the device (metal_device_open()), mapped the 2 IO regions (metal_device_io_region() and checked their physical base addresses (metal_io_phys()). All looks good.

However, when I try to read or write using metal_io_read32() and metal_io_write32() to either IO region, I only seem to be accessing the 1st region. For example, if I write to IO region 1 it affects the phsyical memory for IO region 0; if I read from IO region 1 it appears to actually reads from IO region 0. Verified with devmem. I've got debug code which prints out the physical address just before I call metal_io_read32() or metal_io_write32() and it is correct.

I should add this is on Linux with QEMU session.

Any ideas what the problem could be?

Move this directory out of libmetal -> https://github.com/OpenAMP/libmetal/tree/main/lib/system/generic/xlnx

where is xsdk/r5_0_bsp/psu_cortexr5_0/include ?

Hi, I have encountered an issue with the metal_io_phys() function on arm 64-bit target when io->page_shift is equal to 64, the result of the following shift operation lets 'offset' unchanged, so 'page' equals 'offset' instead of 0:
unsigned long page = offset >> io->page_shift;

It seems that according to the C standard the behaviour of the right shift operator is undefined when the shift operand is greater or equal to the length in bits of the value to be shifted, so it's not a bug in gcc.

I have corrected the problem with the following code:

static inline metal_phys_addr_t
metal_io_phys(struct metal_io_region *io, unsigned long offset)
{
unsigned long page;
if (io->page_shift >= 8 * sizeof(unsigned long))
page = 0;
else
page = offset >> io->page_shift;
...

Thanks

The math in sleep.h for FreeRTOS seems suspect, I think it might create delays 1,000 times larger than the caller expects:

static inline int __metal_sleep_usec(unsigned int usec)
{
const TickType_t xDelay = usec / portTICK_PERIOD_MS;
vTaskDelay(xDelay);
return 0;
}

Shouldn't it be "(usec / 1000) / portTICK_PERIOD_MS" to convert microseconds to ticks?

We currently use printk, but should tie into the proper logging system in zephyr via include/logging/sys_log.h.

I'm getting the following error when building libmetal via bitbake using DEPENDS = "libmetal":

/usr/include/metal/system/linux/sys.h:60:12: error: 'PATH_MAX' undeclared here (not in a function)

It seems like the reason for the error is that linux/sys.h in libmetal includes limits.h, but PATH_MAX is defined in linux/limits.h.

Is it supported to run libmetal on Linux without having the kernel modules uio_pdrv_genirq, uio_dmem_genirq, vfio-pci and uio_pci_generic? The code tries to do modprobe on these in metal_linux_probe_driver. I want to use openamp on Linux but without being tied to those kernel modules.

libmetal is currently relying on a GNU C extension, namely void-ptr arithmetic, which is a violation across all ISO C standards.
Worse, this makes libmetal non-edible by Clang.

libmetal/lib/io.c:54:7: error: wrong type argument to increment [-Werror=pointer-arith]
libmetal/lib/io.c:58:39: error: pointer of type ‘void *’ used in arithmetic [-Werror=pointer-arith]
libmetal/lib/io.c:62:23: error: wrong type argument to increment [-Werror=pointer-arith]
libmetal/lib/io.c:90:7: error: wrong type argument to increment [-Werror=pointer-arith]
libmetal/lib/io.c:94:10: error: pointer of type ‘void *’ used in arithmetic [-Werror=pointer-arith]
libmetal/lib/io.c:97:30: error: wrong type argument to increment [-Werror=pointer-arith]

kind regards,
Mark

The issue is created to record the const aspect of #156. The motivations remain as that issue.

Some points on the effect of using const in place of #define

• Symbols for the constants will be included in library. This increases the size of the library.
• This is only of concern in systems with both using dynamic linkage and with tight space constraints.
• There are 640 #defines (in total) in the codebase. This should bound the upper limit on the effect on size to a few kB.

Many of the constants could better be represented as enums, for increased type safety. This has a larger impact on codebases using libmetal, so such a refactoring should be considered as a seperate issue.

Hello,
We were using OpenAMP within the XIlinx tools 2016.2 and compiling it with g++.
Now we updated the tools versions to 2017.2 and noticed that OpenAMP is not compiling with g++ any more, because of the atomic's present on libmetal.
We have packed it as separate project (wrapper) to compile with gcc and so be able to link it with our g++ compiled project.
Is there going to be a g++ supported implementation? When yes, in which version of Xilinx tools will it be supported?
Thanks

I am having trouble running the libmetal demo on Linux. It seems the IPI device is not defined.
During boot the drivers say the IPI device is not defined.

Here is the boot message related to the remoteproc initialization:
[ 5.099446] zynqmp_r5_remoteproc ff9a0100.zynqmp_r5_rproc: RPU core_conf: split0
[ 5.106917] zynqmp_r5_remoteproc ff9a0100.zynqmp_r5_rproc: IPI resource is not specified.
[ 5.116141] remoteproc remoteproc0: ff9a0100.zynqmp_r5_rproc is available
[ 5.122966] remoteproc remoteproc0: Note: remoteproc is still under development and considered experimental.
[ 5.132820] remoteproc remoteproc0: THE BINARY FORMAT IS NOT YET FINALIZED, and backward compatibility isn't yet guaranteed.
[ 5.144174] zynqmp_r5_remoteproc ff9a0200.zynqmp_r5_rproc: RPU core_conf: split1
[ 5.152017] remoteproc remoteproc1: ff9a0200.zynqmp_r5_rproc is available
[ 5.158828] remoteproc remoteproc1: Note: remoteproc is still under development and considered experimental.
[ 5.168691] remoteproc remoteproc1: THE BINARY FORMAT IS NOT YET FINALIZED, and backward compatibility isn't yet guaranteed.

When loading the R5 code (linmetal-AMP-demo) it emits this:
[ 20.922576] remoteproc remoteproc0: powering up ff9a0100.zynqmp_r5_rproc
[ 20.931915] remoteproc remoteproc0: Booting fw image libmetal-demo, size 541332
[ 20.941412] zynqmp_r5_remoteproc ff9a0100.zynqmp_r5_rproc: RPU boot from TCM.
[ registering: 0, name=ff310000.ipi
registering: 1, name=3ed80000.shm
registering: 2, name=ff110000.ttc

SERVER> ====== libmetal demo: shared memory ======

SERVER> Wait for shared memory demo to start.
20.951156] remoteproc remoteproc0: remote processor ff9a0100.zynqmp_r5_rproc is now up

The libmetal demo common.h file has this:

/* Devices names */
#define BUS_NAME "generic"
#define IPI_DEV_NAME "ff310000.ipi"
#define SHM_DEV_NAME "3ed80000.shm"
#define TTC_DEV_NAME "ff110000.ttc"

My linux demo application common.h files has this:
#define BUS_NAME "platform"
#define IPI_DEV_NAME "ff340000.ipi"
#define SHM_DEV_NAME "3ed80000.shm"
#define TTC_DEV_NAME "ff110000.timer"

It seem the R5 app is using a different address for IPI than the Linux app. My device file (attached) also uses the "ff340000.ipi" definition.

When running the Linux app I get this"
CLIENT> ****** libmetal demo: shared memory ******
metal: info: metal_uio_dev_open: No IRQ for device 3ed80000.shm.
CLIENT> Setting up shared memory demo.
CLIENT> Starting shared memory demo.
CLIENT> Sending message: Hello World - libmetal shared memory demo
CLIENT> Message Received: Hello World - libmetal shared memory demo
CLIENT> Shared memory demo: Passed.
CLIENT> ****** libmetal demo: atomic operation over shared memory ******
metal: info: metal_uio_dev_open: No IRQ for device 3ed80000.shm.
metal: error: failed to bind ff340000.ipi to uio_pdrv_genirq
metal: error: metal_irq_register: irq fd -1 is less than 0.
CLIENT> ERROR: Failed to open device ff340000.ipi.
CLIENT> ERROR: shared memory atomic demo failed.

Any help would be appreciated

Attached is my device file
system-user.dtsi.txt

Hopefully, more of a question than an issue.

I have 2 separate libraries both using libmetal to access h/w devices (different device each library). So, lets' say libA accesses deviceA and libB accesses deviceB. Now, in some cases, I'm just using libA / deviceA and in some cases I'm just using libB / deviceB. So far so good.

However, I now have an application that wants to use libA and libB, i.e. both deviceA and deviceB. The problem that I see here, is that since I'm using Linux shared libraries, the libmetal library is essentially common, i.e. used by both libA and libB, and as such it is sharing some static data structures.

In particular, I have a problem when I try to do "metal_finish()". For example: metal_finish() in libA works, but when I do metal_finish() in libB I get segmentation fault (because it was already "finished" by libA). I suspect there could be similar issues in other places.

How can work around this? Is there a correct way to handle this scenario?

Thanks.

I'm having problems getting a super basic C++ applet with Libmetal to compile in Petalinux 2019.1 for the RFSoC.

My Super Simple Example app is:

#include <iostream>              
#include <metal/io.h> 
                                                                                                                                                                                                                                           
int main(int argc, char *argv[]){                                                                                                    
                                                                                                                                     
   // struct metal_device * device;  
   int i;    

   std::cout << "Hello World!" << std::endl;     

                                                                           
   return 0;                                                                                                                       
}

This build fine in 2018.3 but now in 2019.1 It throws a number of errors relating to atomic:

/usr/include/metal/io.h:42:5: error: 'memory_order' has not been declared
     memory_order order,
     ^~~~~~~~~~~~
/usr/include/metal/io.h:47:6: error: 'memory_order' has not been declared
      memory_order order,
      ^~~~~~~~~~~~
/usr/include/metal/io.h:52:5: error: 'memory_order' has not been declared
     memory_order order,
     ^~~~~~~~~~~~
/usr/include/metal/io.h:57:6: error: 'memory_order' has not been declared
      memory_order order,
      ^~~~~~~~~~~~
/usr/include/metal/io.h:62:6: error: 'memory_order' has not been declared
      memory_order order,
      ^~~~~~~~~~~~
/usr/include/metal/io.h:235:8: error: 'memory_order' has not been declared
        memory_order order, int width)
        ^~~~~~~~~~~~
/usr/include/metal/io.h: In function 'uint64_t metal_io_read(metal_io_region*, long unsigned int, int, int)':
/usr/include/metal/io.h:241:25: error: 'atomic_uchar' was not declared in this scope
  else if (ptr && sizeof(atomic_uchar) == width)
                         ^~~~~~~~~~~~
/usr/include/metal/io.h:241:25: note: suggested alternative:
In file included from /usr/include/metal/atomic.h:21,
                 from test.cpp:2:
/usr/include/c++/8.2.0/atomic:877:34: note:   'std::atomic_uchar'
   typedef atomic<unsigned char>  atomic_uchar;
                                  ^~~~~~~~~~~~
In file included from test.cpp:3:
/usr/include/metal/io.h:242:46: error: expected primary-expression before ')' token
   return atomic_load_explicit((atomic_uchar *)ptr, order);
                                              ^
/usr/include/metal/io.h:242:10: error: 'atomic_load_explicit' was not declared in this scope
   return atomic_load_explicit((atomic_uchar *)ptr, order);
          ^~~~~~~~~~~~~~~~~~~~
/usr/include/metal/io.h:242:10: note: suggested alternative:
In file included from /usr/include/metal/atomic.h:21,
                 from test.cpp:2:
/usr/include/c++/8.2.0/atomic:1089:5: note:   'std::atomic_load_explicit'
     atomic_load_explicit(const volatile atomic<_ITp>* __a,
...

Is anyone else able to replicate the behavior I'm seeing?

Hi there,
I am trying to build the driver for AMD Xilinx AIE and it needs the shmem-provider.h file from libmetal. Currently that file doesn't exist and there is only shmem.h. I can imagine the file was renamed at some point or its funcionality might have been merged into another file... which version of libmetal has this file?

Thank you.

Travis Ci test is no more functional. the Zephyr build fails due to travis environment that is not up to date

A first try to fix it by migrating script environment on Ubuntu bionic seems solve a part of the issue but Zephyr build still fails.

Suggestion from @galak is to abandon the travis CI test an to rewrite it in a github action

I got the following errors while using OpenAmp in RZG2L SMARC Linux Image.

ERROR: libmetal-2018.10-r0 do_fetch: Fetcher failure: Unable to find revision 89276b5 in branch master even from upstream
ERROR: libmetal-2018.10-r0 do_fetch: Fetcher failure for URL: 'git://github.com/OpenAMP/libmetal.git;protocol=https;branch=master'. Unable to fetch URL from any source.

How to do for this??

Hello,

This issue is a copy past of discussion around memory management in OpenAMP and libmetal.
Aim of this issue is to share a proposal to offer an unique libmetal API for the memory management

Nothing very complex, just a proposal to hand over memory management in libmetal porting layer.

Memory management in term of feature:

• variable/structure used in openamp:
  o Dynamically allocated
  o Statically allocated
  => not treated here but should could treated using a compilation switch.
• Shared memory to map.
  o Allocate and/or register a memory region
  o map/unmap of a memory region
  o read/write/set access to the memory

Shared memory access type:

• basic memory mapping (no MMU /IOMMU, no cache…)
   va = pa
• MMU/IOMMU with simple a memory translation
   Need to manage translation between va and pa
• MMU/IOMMU, specific buses with specific memory management ( paging, cache…)
   Specific memory management.

Overview of the shared Memory management used by OpenAMP, provided by libmetal

• io region : memory and i/o register access
   provide ops to access to memory
  libmetal/include/io.h
• dev_mem : memory device access through bus abstraction
  libmetal/include/device.h
  =>based on io_region
• Shmem: Registration and open of shared memory region
  libmetal/include/shmem.h
  => based on io region.
• cache management: used in OpenAMP for flush and invalidate cache
  => libmetal/include/cache.h

Memory management in rpmsg, virtio and rproc frameworks implemented in OpenAMP
 Mainly the io_region

Concerns about current implementation:

• Complex memory management is exposed at OpenAMP level for all platforms.
• OpenAMP should be memory management agnostic.
• Libmetal as porting layer should implement associated complexity level.

Proposal:

• Define an unique generic libmetal API for shared memory management.
• Clients(OpenAMP and application) uses this API to access to shared memory.
• Shared memory regions will be registered. Application is in charge of allocating or mapping the memory associated.
• Shared memory is identified by an handle.

API proposed ( first draft under discussion)
 Based on principles described above.

• An unique libmetal API for shared memory management : shmem.h
  => Current devmem /io region and cache APIs for shared memory management no more exposed, but implemented in libmetal depending on platform.
• Proposed API is inspirited from io_region API.
• OpenAMP & application access to shared memory using a memory region handle and an offset.
• Every action on shared memory is done by libmetal.

/** Generic shared memory data structure that could be optimized depending on platform and OS. */
extern struct metal_generic_mem;

/* Example of a declaration to remap it on io memory region*/
struct metal_generic_shmem {
const char *name;
metal_phys_addr_t pa;
metal_phys_addr_t size;
void * metal_io_region ;
struct metal_list node;
};

/* memory region Registration( carveouts, vrings, buffers...) */
extern int metal_shmem_register_generic(struct metal_generic_shmem *shmem);

/* Retrieve registered memory region by name or address */
extern struct metal_generic_mem *metal_shmem_open_by_name(const char *name, size_t size,
struct metal_io_region **io);
extern struct metal_generic_mem *metal_shmem_open_by_addr(metal_phys_addr_t pa, size_t size,
struct metal_io_region **io);
extern void metal_shmem_block_close(struct metal_generic_shmem *shmem);

/* Get physical address */
extern metal_phys_addr_t pa metal_shmem_to_phy (struct metal_generic_shmem * shmem, unsigned long offset);

/* Memory region access */
extern uint64_t metal_shmem_read(struct metal_generic_mem *mem, unsigned long offset, int width);
extern int metal_shmem_write(struct metal_generic_shmem *shmem, unsigned long offset, uint64_t value, int width);
extern int metal_shmem_block_read(struct metal_generic_shmem *shmem, unsigned long offset, void *restrict dst, int len);
extern int metal_shmem_block_write(struct metal_generic_shmem *shmem, unsigned long offset, const void *restrict src, int len);
extern int metal_shmem_block_set(struct metal_generic_shmem *shmem, unsigned long offset, unsigned char value, int len);

BR,
Arnaud

Examples and demos should be moved to new repostiory: openamp-system-reference.

This should not cause any backward compatibility issues.

Issue Description:

When the application needs to shared data between a device or a remote processor,
it needs to:

• allocate memory for the system
• make the allocated shared memory accessible for both the applicaiton and the device/remote processor
• push and retrieve data from the shared memory

Libmetal APIs should provide the abtraction to allocate and return the shared memory across different operating systems, different remote devices and different types of memories.
The application on top of libmetal doesn't need to care about the underline platform/operating system specific memory management.

Operating Systems:

• Linux(userspace)
• Baremetal
• Zephyr (future)
• ...

Memory Type:

• Normal memory (e.g. DDR)
• Memories over PCIe (future)
• ...

Proposal

Libmetal Components involved:

• metal_io_region
• metal_shm
• metal_device

metal_io_region:

It is the minimum unit to present memory mapped address space, and it abstract how to access the address space. Will keep as what it is now.

metal_shm:

• Today, it is only used to keep track of all the registered metal_io_regions which can be used as shared memory, and allow user to get a metal_io_region from a name.

• Will need to extend this API to:

  • enable allocating shared memory from system or device
    • After the allocating shared memory operation, the application can access the memory.
    • enable assigning the shared memory to a remote device/processor
    • This API will not include the notification to the remote to tell which memory is assigned to Application/driver can use IPC for notification. E.g. RPMsg can be used to inform the remoteprocessor.
    • After assigning the shared memory to a remote:
    • the remote should be able to access memory.
    • device address of the shared memory should be available, so that application can use it to set up DMA or notify remote.

metal_device:

It is a device abstraction for libmetal application.

• Today, it abstracts the memory mapped address regions and operation to open() -- enable user to access its memory mapped address regions and close() operations.
• Will need to add the abtraction to allow to allocate/free memory from the device, and attach/detach memory to the device

Here are the use cases we are considering for now:

• UC1:

  • Linux/Baremetal application without IOMMU needs to share DDR memory to a device/coprocessor.

• UC2:

  • Linux application with IOMMU enabled needs to share DDR memory to a device/coprocessor.

Here is the proposed APIs:

• struct metal_generic_shm *metal_shm_allocate(size_t size, struct metal_device *src_dev, metal_shm_type shm_type);
  • This function will allocate size of memory from src_dev. The shm_type indicate if it is contiguous memory or not, can extend if required.
• struct metal_generic_shm *metal_shm_attach(struct metal_generic_shm *src_shm, struct metal_device *dst_dev);
  • attach the shared memory to the destination address
  • this operation will make the shared memory available to the remote. (It may map the memory differently to the source device in order to have the destination device to access it.)
  • the io_region will contain the memory address which applicaiton can be used to setup DMA or notify remote.
• metal_shm_detach(struct metal_generic_shm *dst_shm, struct metal_device *dst_dev);
  • detach the memory from the destination device
• metal_shm_free(struct metal_generic_shm *src_shm, struct metal_device *src_dev);
  • free the memory from the source device

Here is the proposal on Linux:

• Without IOMMU

  /* Step 1: Allocate shm with metal_shm_allocate() by passing source metal_device pointer.
             e.g. The source device can be ION device. It gets the DMA buf from the device.
  		   We suggest to use ION device so that libmetal can have generic implementaion
  		   to get DMA buffer.
  */
  /* Step 2: attach shm to the target device with metal_shm_attach() by passing dst_dev pointer.
                  the dest device underline Linux kernel driver will import the DMA buf and map it 
                      to make sure it is available for the remote.
  		We propose to add importing DMA buffer to UIO driver, so that the libmetal for
  		Linux can have generic implementation for DMA import.
   */
  
  

• With IOMMU

  /* Step 1: Allocate shm with metal_shm_allocate() by passing NULL src_dev pointer.
             This step will allocate some space with mmap(). And returns shm object
  		   which contains the io_region for the allocated memory.
  */
  /* Step 2: attach shm with vfio device with metal_shm_attach() by passing vfio dst_dev pointer.
             This step will call vfio iotl to map the memory for the device.
   */
  

I ran the libmeal demo that comes prepackaged with PetaLinux 2016.3 and it failed to open an IPI Device

root@Xilinx-ZCU102-2016_3:#
root@Xilinx-ZCU102-2016_3:# libmetal-demo
metal: warning: skipped page size 2097152 - invalid args
metal: error: device platform:ff340000.ipi not found
metal: error: metal_irq_register: irq fd -1 is less than 0.
CLIENT> ERROR: Failed to open device ff340000.ipi.
root@Xilinx-ZCU102-2016_3:~#

I had the libmetal demo running on the R5 as well and could see that it registered the devices properly and was waiting for an interrupt for Linux.

What is the proper way to run this demo?

Hello, and sorry in advance for the vague description. We use libmetal (metal_io_virt) to access shared memory on an Aarch64 Xilinx board, and when compiled with ASAN, we sometimes get strange alternating patterns of 8 bytes 0x0 then 8 bytes 0xff instead of the buffer data we usually get.

Is there anything ringing a bell about libmetal and ASAN interacting strangely?

Regards

The first argument to atomic_compare_exchange_strong should be a pointer to an Atomic variable. However the uses of it in condition.c do not conform to that.

I tried to compile libmetal for zynq linux ,but it fails, show that:

➜  build-libmetal git:(master) ✗ cmake .. -DCMAKE_TOOLCHAIN_FILE=../cmake/platforms/zynq7-linux.cmake
-- Build type:  Debug
-- Host:    Linux/x86_64
-- Target:  Linux/arm
-- Machine: Generic
-- Could NOT find Doxygen (missing:  DOXYGEN_EXECUTABLE)
-- Could NOT find LIBSYSFS (missing:  LIBSYSFS_LIBRARY)
CMake Error at /usr/share/cmake-3.5/Modules/FindPackageHandleStandardArgs.cmake:148 (message):
  Could NOT find LibRt (missing: LIBRT_LIBRARIES)
Call Stack (most recent call first):
  /usr/share/cmake-3.5/Modules/FindPackageHandleStandardArgs.cmake:388 (_FPHSA_FAILURE_MESSAGE)
  cmake/modules/FindLibRt.cmake:36 (find_package_handle_standard_args)
  cmake/depends.cmake:24 (find_package)
  CMakeLists.txt:18 (include)


-- Configuring incomplete, errors occurred!
See also "/home/osboxes/src/tool/libmetal/build-libmetal/CMakeFiles/CMakeOutput.log".
See also "/home/osboxes/src/tool/libmetal/build-libmetal/CMakeFiles/CMakeError.log".

I use arm-linux-gnueabihf-gcc as cross-compiler,and I have modified it in zynq7-linux.cmake.
How to solve this problem?

Hi!,
I'm testing the libmetal/OpenAMP on ZynqMP (ZU3EG) with Xilinx SDK 2018.2 (patched with fix for urandom read seed #64)

I've succesfull run the "kernel space echo test" and now I'm tring to do same test from user space.

I've extended the DTS like below but the test-metal-shared fail with this error:

# test-metal-shared
metal: debug:     added page size 4096 @/tmp
metal: debug:     registered platform bus
metal: info:      running [atomic]
metal: info:      result [atomic]............................ pass
metal: info:      running [mutex]
metal: info:      result [mutex]............................. pass
metal: info:      running [shmem]
metal: warning:   failed to mlock shmem - Cannot allocate memory
metal: warning:   failed to mlock shmem - Cannot allocate memory
metal: warning:   failed to mlock shmem - Cannot allocate memory
metal: warning:   failed to mlock shmem - Cannot allocate memory
metal: warning:   failed to mlock shmem - Cannot allocate memory
metal: warning:   failed to mlock shmem - Cannot allocate memory
metal: error:     pagemap page not present, 3fd80f40 -> 0
metal: error:     pagemap page not present, 3fd80f48 -> 0
...
...
...

Any suggestions? Thanks in advance.

--
Francesco.

DTS:

/ {
    reserved-memory {
        #address-cells = <2>;
        #size-cells = <2>;
        ranges;
        rproc_0_reserved: rproc@40000000 {
            no-map;
            /*
            0x40000000 elf0     (len: 0x2000000)
            0x42000000 elf1     (len: 0x2000000)
            0x44000000 ring0_tx (len: 40000)
            0x44040000 ring0_rx (len: 40000)
            0x44100000 ring1_tx (len: 40000)
            0x44140000 ring1_rx (len: 40000)
            0x44200000 shm0     (len: 0x100000)
            0x44300000 shm1     (len: 0x100000)
            */
            reg = <0x0 0x40000000 0x0 0x4400000>;
        };
    };

    power-domains {
        pd_r5_0: pd_r5_0 {
            #power-domain-cells = <0x0>;
            pd-id = <0x7>;
        };
        pd_r5_1: pd_r5_1 {
            #power-domain-cells = <0x0>;
            pd-id = <0x8>;
        };
        pd_tcm_0_a: pd_tcm_0_a {
            #power-domain-cells = <0x0>;
            pd-id = <0xf>;
        };
        pd_tcm_0_b: pd_tcm_0_b {
            #power-domain-cells = <0x0>;
            pd-id = <0x10>;
        };
        pd_tcm_1_a: pd_tcm_1_a {
            #power-domain-cells = <0x0>;
            pd-id = <0x11>;
        };
        pd_tcm_1_b: pd_tcm_1_b {
            #power-domain-cells = <0x0>;
            pd-id = <0x12>;
        };
    };

    amba {
        r5_0_tcm_a: tcm@ffe00000 {
            compatible = "mmio-sram";
            reg = <0 0xFFE00000 0x0 0x10000>;
            pd-handle = <&pd_tcm_0_a>;
        };
        r5_0_tcm_b: tcm@ffe20000 {
            compatible = "mmio-sram";
            reg = <0 0xFFE20000 0x0 0x10000>;
            pd-handle = <&pd_tcm_0_b>;
        };
        r5_1_tcm_a: tcm@ffe90000 {
            compatible = "mmio-sram";
            reg = <0 0xFFE90000 0x0 0x10000>;
            pd-handle = <&pd_tcm_1_a>;
        };
        r5_1_tcm_b: tcm@ffe92000 {
            compatible = "mmio-sram";
            reg = <0 0xFFEB0000 0x0 0x10000>;
            pd-handle = <&pd_tcm_1_b>;
        };


        elf_ddr_0: ddr@40000000 {
            compatible = "mmio-sram";
            reg = <0 0x40000000 0x0 0x2000000>;
        };

        elf_ddr_1: ddr@42000000 {
            compatible = "mmio-sram";
            reg = <0 0x42000000 0x0 0x2000000>;
        };


        test_r50: zynqmp_r5_rproc@0 {
            compatible = "xlnx,zynqmp-r5-remoteproc-1.0";
            reg = <0x0 0xff9a0100 0 0x100>, <0x0 0xff340000 0 0x100>, <0x0 0xff9a0000 0 0x100>;
            reg-names = "rpu_base", "ipi", "rpu_glbl_base";
            dma-ranges;
            core_conf = "split0";
            srams = <&r5_0_tcm_a &r5_0_tcm_b &elf_ddr_0>;
            pd-handle = <&pd_r5_0>;
            interrupt-parent = <&gic>;
            interrupts = <0 29 4>;

        } ;
       test_r51: zynqmp_r5_rproc@1 {
            compatible = "xlnx,zynqmp-r5-remoteproc-1.0";
            reg =<0x0 0xff9a0200 0 0x100>, <0x0 0xff340000 0 0x100>, <0x0 0xff9a0000 0 0x100>;
            reg-names = "rpu_base", "ipi", "rpu_glbl_base";
            dma-ranges;
            core_conf = "split1";
            srams = <&r5_1_tcm_a &r5_1_tcm_b &elf_ddr_1>;
            pd-handle = <&pd_r5_1>;
            interrupt-parent = <&gic>;
            interrupts = <0 29 4>;

        } ;

		vring0: vring@0 {
			compatible = "vring_uio";
			reg = <0x0 0x44000000 0x0 0x40000>;
		};
		shm0: shm@0 {
			compatible = "shm_uio";
			reg = <0x0 0x44200000 0x0 0x80000>;
		};
		
		vring1: vring@1 {
			compatible = "vring_uio";
			reg = <0x0 0x44100000 0x0 0x40000>;
		};
		shm1: shm@1 {
			compatible = "shm_uio";
			reg = <0x0 0x44300000 0x0 0x80000>;
		};
		
		ipi0: ipi@0 {
			compatible = "ipi_uio";
			reg = <0x0 0xff340000 0x0 0x1000>;
			interrupt-parent = <&gic>;
			interrupts = <0 29 4>;
		};

    };
};

Hello, I'm using petalinux 2018.3 and trying to build a custom application that uses the Zynq RF Data Converter which relies on the libmetal library. When I build my application with my RFADC.h header file above a #include future then I get the following error:

`In file included from /opt/petalinux/2018.3/sysroots/aarch64-xilinx-linux/usr/include/metal/atomic.h:21:0,
from /opt/petalinux/2018.3/sysroots/aarch64-xilinx-linux/usr/include/metal/io.h:21,
from /opt/petalinux/2018.3/sysroots/aarch64-xilinx-linux/usr/include/metal/device.h:16,
from /opt/petalinux/2018.3/sysroots/aarch64-xilinx-linux/usr/include/xrfdc.h:214,
from include/RFADC.h:27,
from include/ZCU111DevScanner.h:32,
from src/ZCU111DevScanner.cpp:12:
/opt/petalinux/2018.3/sysroots/aarch64-xilinx-linux/usr/include/c++/7.3.0/future:320:43: error: use of deleted function ‘std::atomic_flag::atomic_flag(const std::atomic_flag&)’
atomic_flag _M_retrieved = ATOMIC_FLAG_INIT;

In file included from /opt/petalinux/2018.3/sysroots/aarch64-xilinx-linux/usr/include/c++/7.3.0/bits/shared_ptr_atomic.h:33:0,
from /opt/petalinux/2018.3/sysroots/aarch64-xilinx-linux/usr/include/c++/7.3.0/memory:82,
from /opt/petalinux/2018.3/sysroots/aarch64-xilinx-linux/usr/include/c++/7.3.0/thread:39,
from ../blackmore_common/include/CommonMisc.h:24,
from ../blackmore_common/include/BlackmoreMessage.h:16,
from include/ZCU111DevScanner.h:13,
from src/ZCU111DevScanner.cpp:12:
/opt/petalinux/2018.3/sysroots/aarch64-xilinx-linux/usr/include/c++/7.3.0/bits/atomic_base.h:164:5: note: declared here
atomic_flag(const atomic_flag&) = delete;
^~~~~~~~~~~
/opt/petalinux/2018.3/sysroots/aarch64-xilinx-linux/usr/include/c++/7.3.0/bits/atomic_base.h:169:15: note: after user-defined conversion: constexpr std::atomic_flag::atomic_flag(bool)
constexpr atomic_flag(bool __i) noexcept
^~~~~~~~~~~

`

But if I move the #include RFADC.h to be below the #include future then it builds successfully. I understand that this works, but it seems like a hack and I cannot count on someone to put the header files in the proper order in the future.

Is there any way to solve this so that the application will build successfully independently on where the header files are (with regards to libmetal)? Thanks in advance.

Now that metal_softirq_enabled is an atomic_char we can't just directly access metal_softirq_enabled in metal_softirq_dispatch()

Getting the following build error:

/home/galak/git/zephyr/ext/hal/libmetal/libmetal/lib/softirq.c: In function ‘metal_softirq_dispatch’:
/home/galak/git/zephyr/ext/hal/libmetal/libmetal/lib/softirq.c:93:32: error: invalid operands to binary != (have ‘atomic_char {aka struct <anonymous>}’ and ‘int’)
   if (metal_softirq_enabled[i] != 0 &&
       ~~~~~~~~~~~~~~~~~~~~~~~~ ^~
make[2]: *** [zephyr/ext/hal/libmetal/libmetal/lib/CMakeFiles/metal.dir/build.make:154: zephyr/ext/hal/libmetal/libmetal/lib/CMakeFiles/metal.dir/softirq.c.obj] Error 1
make[2]: *** Waiting for unfinished jobs....

how to solve ?

XLNX_PLATFORM flag is not passed to libmetal clients such as OpenAMP, they need to define it to compile with current libmetal. Instead fix this in libmetal in a way where libmetal clients do not need to define this flag.

Hello everyone,
I am just starting with libmetal and openamp so pardon me if this question is very basic.

So I need to compiled some of the examples in openamp, especially echo one. In order to run it on RPC I have to compiled using the baremetal compilation method. I am working on RedPitaya which zynq7 based device so I am doing the following:

$ cmake .. -DCMAKE_TOOLCHAIN_FILE=../cmake/platforms/zynq7_generic.cmake                                                                                                               

-- Host:    Linux/x86_64
-- Target:  Generic/arm
-- Machine: zynq7
-- C_FLAGS : -mcpu=cortex-a9 -mfpu=vfpv3 -mfloat-abi=hard -Wall -Wextra
-- Found LIBMETAL: /usr/local/lib/libmetal.so  
-- Looking for include file stdatomic.h
-- Looking for include file stdatomic.h - found
-- Looking for include file fcntl.h
-- Looking for include file fcntl.h - found
-- Configuring done
-- Generating done
-- Build files have been written to: /home/user/open-amp/build-baremetal

Then I do the make install but it is complaining about not finding some of the xilinx files (xscugic.h) . I don't know whether these files are generated when we import an hdf files or these are just generic files. So do we need to modify the cmake files to include some other directories or whats the method here? Below is the result of running make:

$ make VERBOSE=1 DESTDIR=install_dir/ install
/home/waqar/tools/Xilinx/SDK/2017.2/tps/lnx64/cmake-3.3.2/bin/cmake -H/home/user/libmetal/libmetal -B/home/user/libmetal/libmetal/build-baremetal --check-build-system
 CMakeFiles/Makefile.cmake 0
/home/waqar/tools/Xilinx/SDK/2017.2/tps/lnx64/cmake-3.3.2/bin/cmake -E cmake_progress_start /home/user/libmetal/libmetal/build-baremetal/CMakeFiles /home/waqar/workspace/MI
T/libmetal/build-baremetal/CMakeFiles/progress.marks
make -f CMakeFiles/Makefile2 all
make[1]: Entering directory '/home/user/libmetal/libmetal/build-baremetal'
make -f lib/CMakeFiles/metal-static.dir/build.make lib/CMakeFiles/metal-static.dir/depend
make[2]: Entering directory '/home/user/libmetal/libmetal/build-baremetal'
cd /home/user/libmetal/libmetal/build-baremetal && /home/waqar/tools/Xilinx/SDK/2017.2/tps/lnx64/cmake-3.3.2/bin/cmake -E cmake_depends "Unix Makefiles" /home/waqar/workspa
ce/MIT/libmetal /home/user/libmetal/libmetal/lib /home/user/libmetal/libmetal/build-baremetal /home/user/libmetal/libmetal/build-baremetal/lib /home/waqar/works
pace/MIT/libmetal/build-baremetal/lib/CMakeFiles/metal-static.dir/DependInfo.cmake --color=
make[2]: Leaving directory '/home/user/libmetal/libmetal/build-baremetal'
make -f lib/CMakeFiles/metal-static.dir/build.make lib/CMakeFiles/metal-static.dir/build
make[2]: Entering directory '/home/user/libmetal/libmetal/build-baremetal'
[  5%] Building C object lib/CMakeFiles/metal-static.dir/dma.c.obj
cd /home/user/libmetal/libmetal/build-baremetal/lib && /home/waqar/tools/Xilinx/SDK/2017.2/gnu/aarch32/lin/gcc-arm-none-eabi/bin/arm-none-eabi-gcc  -DDEFAULT_LOGGER_ON -DME
TAL_INTERNAL -mcpu=cortex-a9 -mfpu=vfpv3 -mfloat-abi=hard -g -I/home/user/libmetal/libmetal/build-baremetal/lib/include    -Wall -Werror -Wextra -o CMakeFiles/metal-static.
dir/dma.c.obj   -c /home/user/libmetal/libmetal/lib/dma.c
In file included from /home/user/libmetal/libmetal/build-baremetal/lib/include/metal/system/generic/sys.h:28:0,
                 from /home/user/libmetal/libmetal/build-baremetal/lib/include/metal/sys.h:81,
                 from /home/user/libmetal/libmetal/build-baremetal/lib/include/metal/io.h:22,
                 from /home/user/libmetal/libmetal/build-baremetal/lib/include/metal/device.h:16,
                 from /home/user/libmetal/libmetal/lib/dma.c:9:
/home/user/libmetal/libmetal/build-baremetal/lib/include/metal/system/generic/./zynq7/sys.h:17:21: fatal error: xscugic.h: No such file or directory
 #include "xscugic.h"
                     ^
compilation terminated.
lib/CMakeFiles/metal-static.dir/build.make:62: recipe for target 'lib/CMakeFiles/metal-static.dir/dma.c.obj' failed
make[2]: *** [lib/CMakeFiles/metal-static.dir/dma.c.obj] Error 1
make[2]: Leaving directory '/home/user/libmetal/libmetal/build-baremetal'
CMakeFiles/Makefile2:93: recipe for target 'lib/CMakeFiles/metal-static.dir/all' failed
make[1]: *** [lib/CMakeFiles/metal-static.dir/all] Error 2
make[1]: Leaving directory '/home/user/libmetal/libmetal/build-baremetal'
Makefile:138: recipe for target 'all' failed
make: *** [all] Error 2

Cheers.
Waqar

__builtin_assume_aligned() is gnu gcc only, need to have a macro for it.

Commit 10a0d5b removes the priority inheritance mechanism FreeRTOS is using with its mutexes.
Suggest to revert this commit.

Having CMAKE_BUILD_TYPE be the empty string is a valid configuration. But options.cmake will change it to Debug when it is empty string, thereby preventing us from using this valid configuration.

https://cmake.org/cmake/help/v3.17/variable/CMAKE_BUILD_TYPE.html

This statically specifies what build type (configuration) will be built in this build tree. Possible values are empty, Debug, Release, RelWithDebInfo, MinSizeRel, …

libmetal was designed to make Linux user space and RTOS and bare-metal have the same API.
This works and the amount of abstraction present is required for Linux user space (uio & vfio etc).

HOWEVER, when viewed from the perspective of only bare-metal and RTOS the library is heavy weight and overly abstract.

We should find a way to skinny down libmetal for the common case of bare-metal and RTOS. Some people have suggested a different code base with the same API but we should also look at a common code base with A compiler time flag to select the model desired. Make the default the simple model for bare-metal and RTOS platforms.

Evaluation criteria:

• Code & data size
• Dynamic memory allocation:
  • can be made optional?
  • simple allocate only possible?
• Ease of porting to a new platform on a supporrted RTOS
  • This should be zero work, prove this is true
  • the library is pre-ported to Zephyr FreeRTOS Nuttx etc, If the RTOS works on a given platfrom libmetal should as well.
• Ease of porting to a new bare-metal platform
  • W/o the abstraction of an RTOS, some porting work is required
  • Make this as simple as possible
  • Be more prescriptive about what the bare-metal BSP provides. Give porters a clean target to fit.

Is there a reason a new error code, METAL_ERANGE, was not created instead of using -ERANGE?

I see you have the correct change in 2371f61 , but in main branch libmetal fail to build because of include is using zepher/kernel.h

lib/system/zephyr/mutex.g is an example.

/Tiru

The README is a bit log and rambling.

The README is also the main page for the doygen docs which make it one big page also.

Issues/Actions

• Library categogy data is good but should be in the intro to each section
• • move this content into per module files in doc/ and replace by a table in README.
• Contributing info does not need to be on the main page, make a page and link it from README
• Simply build instructions on the main page (README) and move the detailed instructions to a sub-page.
• • Example1: Most people that build for Zephyr will use the Zephyr integration and it will just work. Move the details for the other 5% off the main page.
• • Example2: Build for bare-metal, use an existung generic target on main page. Move instucions on make a new custom target to new page and suggest they carrry that def in their build system instead of pushing to libmetal.
• (Example2b: ceva CPU port is just generic. Make it easy to support a new CPU that is generic.)

Would be nice to have that option. Something using mprotect for SHM devices, for example.

The CI is passed but with the following error in traces ( error is not taken into account

Running tests...
/github/workspace/.venv/lib/python3.12/site-packages/cmake/data/bin/ctest --force-new-ctest-process
Test project /github/workspace/build-linux
Start 1: test-metal-shared
1/2 Test #1: test-metal-shared ................***Failed 0.03 sec
Start 2: test-metal-static
2/2 Test #2: test-metal-static ................***Failed 0.03 sec

0% tests passed, 2 tests failed out of 2

Adding traces in libmetal code show that an error occurs in metal_linux_probe_driver() failing to install following kernel modules:

• uio_pdrv_genirq
• uio_dmem_genirq
• vfio-pci
• uio_pci_generic

Seems that these modules are not available using venv

$ modprobe uio_pdrv_genirq
modprobe: FATAL: Module uio_pdrv_genirq not found in directory /lib/modules/6.5.0-1018-azure

In libmetal/lib/system/freertos/sleep.h

static inline int __metal_sleep_usec(unsigned int usec)
{
	const TickType_t xDelay = usec / portTICK_PERIOD_MS;
	vTaskDelay(xDelay);
	return 0;
}

If the configTICK_RATE_HZ of is bigger than 1000 the portTICK_PERIOD_MS is 0 and you get division by zero (not recommended).
Better use seconds * configTICK_RATE_HZ to calculate how many ticks to sleep.

in sys,c, function sys_irq_save_disable() the code shows
value = mfcpsr() & XIL_EXCEPTION_ALL;

If interrupts are enabled by default, value will always result in 0x0.

For example say CPSR = 6000001F (both IRQ and FIR disable bits are low)

0x6000001F & (0x40 | 0x80) wiil yield 0x0000_000

Because int_old_value is initialized to 0x0000_0000 the proceeding If statement will be false and the branch not taken - resulting in the enables remaining ON instead of being turned off

Running test-metal-shared and get a failed result

root@localhost:~/libmetal/build/test# sudo ./test-metal-shared 
libhugetlbfs: WARNING: Unable to find default kernel huge page size
metal: debug:     Failed fread /dev/urandom
metal: debug:     added page size 4096 @/tmp
metal: debug:     registered platform bus
metal: info:      running [atomic]
metal: info:      result [atomic]............................ pass
metal: info:      running [mutex]
metal: info:      result [mutex]............................. pass
metal: info:      running [shmem]
metal: info:      result [shmem]............................. pass
metal: info:      running [condition]
metal: info:      result [condition]......................... pass
metal: info:      running [spinlock]
metal: info:      result [spinlock].......................... pass
metal: info:      running [alloc]
metal: info:      result [alloc]............................. pass
metal: info:      running [irq]
metal: error:     unregister irq 3 match handler success but fail expected
metal: info:      result [irq]............................... fail - error: Invalid argument
metal: info:      running [version]
metal: debug:     metal_linux_irq_handling: poll unexpected. fd 8: 32
metal: info:      result [version]........................... pass
metal: debug:     metal_linux_irq_shutdown
metal: debug:     metal_linux_irq_handling: poll unexpected. fd 8: 32
metal: debug:     unregistered platform bus

my dts:

/ {
    compatible = "xlnx,zynq-7000";

    reserved-memory {
        #address-cells = <1>;
        #size-cells = <1>;
        /* Reserved memory for both firmware and shared memory */
        rproc_0_reserved: rproc@3ed000000 {
            no-map;
            reg = <0x18000000 0x8000000>;
        };
    };

 
    amba {
	 remoteproc0: remoteproc@0 {
	 	compatible = "xlnx,zynq_remoteproc";
	 	reg = <0x00000000 0x10000000>;
	 	firmware = "firmware";
	 	vring0 = <15>;
	 	vring1 = <14>;
	 };
        shm0: shm@0 {
            compatible = "shm_uio";
            reg = <0x0 0x1ed80000 0x0 0x80000>;
        };
 
        ipi0: ipi@0 {
            compatible = "ipi_uio";
            reg = <0x0 0xff340000 0x0 0x1000>;
            interrupt-parent = <&intc>;
            interrupts = <0 29 4>;
        };
 
 
    };
};

For a lot of embedded targets, libmetal would be included into a much larger project that might not use cmake.
It would be good if the code didn't use configure_file symbols in the code like PROJECT_SYSTEM.
One option could be to change the include path passed to the compiler instead of this.

Unlike open-amp CI, this script uses the very latest version of the Zephyr SDK on each PR check.
This is bad. While it is good to test against the latest SDK, that should not be mixed into checking a PR.

The default PR should use a pinned version of the SDK (and the same one as open-amp).

It would be good to have separate CI jobs that would test a known good library (current main should be fine) against the latest Zephyr SDK, latest Zephyr, latest Ubuntu etc

I am cross compiling libmetal for bare metal but fail.
One of the many similar error messages:

D:\Xilinx\SDK\2019.1\gnu\aarch32\nt\gcc-arm-none-eabi\bin\arm-none-eabi-gcc.exe -DDEFAULT_LOGGER_ON -DMETAL_INTERNAL -ID:/repo/libmetal/build/lib/include -Wall -Werror -Wextra -MD -MT lib/CMakeFiles/metal-static.dir/dma.c.obj -MF lib\CMakeFiles\metal-static.dir\dma.c.obj.d -o lib/CMakeFiles/metal-static.dir/dma.c.obj -c D:/repo/libmetal/lib/dma.c
In file included from D:/repo/libmetal/build/lib/include/metal/sys.h:85,
from D:/repo/libmetal/build/lib/include/metal/io.h:22,
from D:/repo/libmetal/build/lib/include/metal/device.h:16,
from D:/repo/libmetal/lib/dma.c:9:
D:/repo/libmetal/build/lib/include/metal/system/generic/sys.h:29:10: fatal error: ./generic/sys.h: No such file or directory
#include "./generic/sys.h"

The generated header in libmetal/build/lib/include/metal/system/generic wants to include a "./generic/sys.h", but there is no libmetal/build/lib/include/metal/system/generic/generic/sys.h. How should I fix this? Should I just add libmetal/build/lib/include/metal/system to the include directories? Thanks.

In several places in the code here, #defines are used to declare functions. Is there a rational for this?

The drawbacks of this style are it:

• removes any semblance of compiler checked type safety,
• makes generating FFI bindings to this library unnecessarily manual (and more error prone). This is certainly true for rust, and probably for python, although I haven't tried that recently.
• reduces IDE tooling support.

I can't think of a rational for using this style - other than maybe you don't trust a compiler to inline - although that seems a pretty low bar.

I'm willing to submit PR(s) to address this, if desired

A related issue would be using consts in place of #defines for constants. The rational is similar, you gain type safety, tooling support, and simplify FFI generation.

Need to improve the CI checkpatch to highlight check report.
the check are generally linked to some coding rules that should be respected for readability

Rename the master branch to main in OpenAMP GitHub repos.

update documentation and CI test according to the new default branch

To be able to check the compatibility of the API with C++ projects, the CI tests should be improved.