Maíra Canal

Unleashing Power: Enabling Super Pages on the RPi

Mon, 28 Oct 2024 12:00:00 +0000

Unleashing the power of 3D graphics in the Raspberry Pi is a key commitment for Igalia through its collaboration with Raspberry Pi. The introduction of Super Pages for the Raspberry Pi 4 and 5 marks another step in this journey, offering some performance enhancements and more efficient memory usage. In this post, we’ll dive deep into the technical details of Super Pages, discuss the challenges we faced during implementation, and illustrate the benefits this feature brings to the Raspberry Pi ecosystem.

What are Super Pages?

A Memory Management Unit (MMU) is a hardware component responsible for handling memory access at the system level. It translates virtual addresses used by programs into physical addresses in main memory, enabling efficient memory management and protection. The MMU allows the operating system to allocate memory dynamically, isolating processes from one another to prevent them from interfering with each other’s memory.

Recommendation: 📚 Structured computer organization by Andrew Tanenbaum

The V3D MMU, which is part of the Broadcom GPU found in the Raspberry Pi 4 and 5, is responsible for translating 32-bit virtual addresses (VA) used by V3D into 40-bit physical addresses used externally to V3D. The MMU relies on a page table, stored in physical memory, which maps virtual addresses to their corresponding physical addresses. The operating system manages this page table, and the MMU uses it to perform address translation during memory access.

A fundamental principle of modern operating systems is that memory is not stored contiguously. Instead, a contiguous block of memory is divided into smaller blocks, called “pages”, which are scattered across the entire address space. These pages are typically 4KB in size. This approach enables more efficient memory management and allows for features like virtual memory and memory protection.

Over the years, the amount of available memory in computers has increased dramatically. An early IBM PC had up to 640 KiB of RAM, whereas the ThinkPad I’m typing on right now has 32 GB of RAM. Naturally, memory demands have grown alongside this increase. Today, it’s common for web browsers to consume several gigabytes of RAM, and a single shader can take up multiple megabytes.

As memory usage grows, a 4KB page size may become inefficient for managing large memory blocks. Handling a large number of small pages for a single block means the MMU must perform multiple address translations, which increases overhead. This can reduce the effectiveness of the Translation Lookaside Buffer (TLB), as it must store and handle more entries, potentially leading to more cache misses and reduced overall performance.

This is why many CPU manufacturers have introduced support for larger page sizes. For instance, x86 CPUs typically support 4KB and 2MB pages, with 1GB pages available if supported by the hardware. Similarly, ARM64 CPUs can support 4KB, 16KB, and 64KB page sizes. These larger page sizes help reduce the number of pages the MMU needs to manage, improving performance by reducing the overhead of address translation and making more efficient use of the TLB.

So, if CPUs are using bigger sizes, why shouldn’t GPUs do the same?

By default, V3D supports 4KB pages. However, by setting specific bits in the page table entry, it is possible to create 64KB “Big Pages” and 1MB “Super Pages.” The issue is that the current V3D driver available in Linux does not enable the use of Big or Super Pages, meaning this hardware feature is currently unused.

The advantage of enabling Big and Super Pages is that once an entry for any page within a Big or Super Page is cached in the MMU, it can be used to translate all virtual addresses within that page’s range without needing to fetch additional entries. In theory, this should result in improved performance, especially for applications with high memory demands, such as those using multiple large buffer objects (BOs).

As Igalia continually strives to enhance the experience for Raspberry Pi users, we decided to implement this feature in the upstream kernel. But before diving into the implementation details, let’s take a look at the real-world results and see if the theoretical benefits of Super Pages have translated into measurable improvements for Raspberry Pi users.

What Does This Feature Mean for RPi Users?

With Super Pages implemented, let’s now explore the actual performance improvements observed on the Raspberry Pi and see how impactful this feature is for users.

Benchmarking Super Pages: Traces and FPS Improvements

To measure the impact of Super Pages, we tested a variety of games and demos traces on the Raspberry Pi 4 and 5, covering genres from action to racing. On average, we observed a +1.40% FPS improvement on the Raspberry Pi 4 and a +1.30% improvement on the Raspberry Pi 5.

For instance, on the Raspberry Pi 4, Warzone 2100 saw an 8.36% FPS increase, and on the Raspberry Pi 5, Quake II enjoyed a 3.62% boost. These examples demonstrate the benefits of Super Pages in resource-demanding applications, where optimized memory handling becomes critical.

Raspberry Pi 4 FPS Improvements

Trace	Before Super Pages	After Super Pages	Improvement
warzone2100.30secs.1024x768.trace	56.39	61.10	+8.36%
ue4_shooter_game_shooting_low_quality_640x480.gfxr	20.71	21.47	+3.65%
quake3e_capture_frames_1800_through_2400_1920x1080.gfxr	60.88	62.50	+2.67%
supertuxkart-menus_1024x768.trace	112.62	115.61	+2.65%
ue4_shooter_game_shooting_high_quality_640x480.gfxr	20.45	20.88	+2.10%
quake2-gles3-1280x720.trace	59.76	60.84	+1.82%
ue4_sun_temple_640x480.gfxr	27.60	28.03	+1.54%
vkQuake_capture_frames_1_through_1200_1280x720.gfxr	54.59	55.30	+1.29%
ue4_shooter_game_low_quality_640x480.gfxr	32.75	33.08	+1.00%
sponza_demo02_800x600.gfxr	20.90	21.03	+0.61%
supertuxkart-racing_1024x768.trace	8.58	8.63	+0.60%
ue4_shooter_game_high_quality_640x480.gfxr	19.62	19.74	+0.59%
serious_sam_trace02_1280x720.gfxr	44.00	44.21	+0.50%
ue4_vehicle_game-2_640x480.gfxr	12.59	12.65	+0.49%
sponza_demo01_800x600.gfxr	21.42	21.46	+0.19%
quake3e-1280x720.trace	84.45	84.52	+0.09%

Raspberry Pi 5 FPS Improvements

Trace	Before Super Pages	After Super Pages	Improvement
quake2-gles3-1280x720.trace	151.77	157.26	+3.62%
supertuxkart-menus_1024x768.trace	306.79	313.88	+2.31%
warzone2100.30secs.1024x768.trace	140.92	144.03	+2.21%
vkQuake_capture_frames_1_through_1200_1280x720.gfxr	131.45	134.20	+2.10%
ue4_vehicle_game-2_640x480.gfxr	24.42	24.88	+1.89%
ue4_shooter_game_high_quality_640x480.gfxr	32.12	32.53	+1.29%
ue4_sun_temple_640x480.gfxr	42.05	42.55	+1.20%
ue4_shooter_game_shooting_high_quality_640x480.gfxr	52.77	53.31	+1.04%
quake3e-1280x720.trace	238.31	240.53	+0.93%
warzone2100.70secs.1024x768.trace	151.09	151.81	+0.48%
sponza_demo02_800x600.gfxr	50.81	51.05	+0.46%
supertuxkart-racing_1024x768.trace	20.91	20.98	+0.33%
ue4_shooter_game_low_quality_640x480.gfxr	59.68	59.86	+0.29%
quake3e_capture_frames_1_through_1800_1920x1080.gfxr	167.70	168.17	+0.29%
ue4_shooter_game_shooting_low_quality_640x480.gfxr	53.40	53.51	+0.22%
quake3e_capture_frames_1800_through_2400_1920x1080.gfxr	163.37	163.64	+0.17%
serious_sam_trace02_1280x720.gfxr	60.00	60.03	+0.06%
sponza_demo01_800x600.gfxr	45.04	45.04	<.01%

While an average +1% FPS improvement might seem modest, Super Pages can deliver more noticeable gains in memory-intensive 3D applications and when the GPU is under heavy usage. Let’s see how the Super Pages perform on Mesa CI.

Benchmarking Super Pages: Mesa CI Job Duration

To avoid introducing regressions in user-space, I usually test my custom kernels with Mesa CI, focusing on the “broadcom-postmerge” stage to verify that all Piglit and CTS tests ran smoothly. For Super Pages, I was pleasantly surprised by the job duration results, as some job durations were reduced by several minutes.

Mesa CI Jobs Duration Improvements

Job	Before Super Pages	After Super Pages
v3d-rpi4-traces:arm64	~4m30s	~3m40s
v3d-rpi5-traces:arm64	~3m30s	~2m45s
v3d-rpi4-gl-full:arm64 */6	~24-25 minutes	~22-23 minutes
v3d-rpi5-gl-full:arm64	~48 minutes	~48 minutes
v3dv-rpi4-vk-full:arm64 */6	~44 minutes	~41 minutes
v3dv-rpi5-vk-full:arm64	~102 minutes	~92 minutes

Seeing these reductions is especially rewarding. For example, the “v3dv-rpi5-vk-full:arm64” job duration decreased by 10 minutes, meaning more FPS for users and shorter wait times for Mesa developers.

Benchmarking Super Pages: PS2 Emulation

After sharing a couple of tables, I’ll admit that showcasing performance improvements solely through numbers doesn’t always convey the real impact. Personally, I find it more satisfying to see performance gains in action with real-world applications.

This led me to explore PlayStation 2 (PS2) emulation on the RPi 5. From watching YouTube videos, I noticed that PS2 is a popular console for the RPi 5. While the PlayStation (PS1) emulates well even on the RPi 4, and Nintendo 64 and Sega Saturn struggle across most hardware, PS2 hits a sweet spot for testing the RPi 5’s limits.

Fortunately, I still have my childhood PS2 — my second console after the Nintendo GameCube, and one of the most successful consoles worldwide, including in Brazil. With a library packed with titles like Metal Gear Solid, Resident Evil, Tomb Raider, and Shadow of the Colossus, the PS2 remains a great system for collectors and retro gamers alike.

I selected a few games from my collection to benchmark on the RPi 5 using a PS2 emulator. My emulator of choice was Aether SX2 with Vulkan support. Although AetherSX2 is no longer in development, it still performs well on the RPi.

Initially, many games were barely playable, especially those with large buffer objects, like Shadow of the Colossus and Gran Turismo 4. However, after enabling Super Pages support, I noticed immediate improvements. For example, Shadow of the Colossus wouldn’t even open before Super Pages, and while it’s not fully playable yet, it does load now. This isn’t a silver bullet, but it’s a step forward in improving the driver one piece at a time.

I ended up selecting four games for a video comparison: Burnout 3: Takedown, Metal Gear Solid 3: Snake Eater, Resident Evil 4, and Tekken 4.

Your browser does not support the video tag.

Disclaimer: The BIOS used in the emulator was extracted from my own PS2, and I played only games I own, with ROMs I personally extracted. Neither I nor Igalia encourage using downloaded BIOS or ROM files from the internet.

From the video, we can see noticeable improvements in all four games. Although they aren’t perfectly playable yet, the performance gains are evident, particularly in Resident Evil 4, where the gameplay saw a solid 5 FPS boost. I realize 18 FPS might not satisfy most players, but I still had a lot of fun playing Resident Evil 4 on the RPi 5.

When tracking the FPS for these games, it’s clear that the performance gains go well beyond the average 1% seen in other benchmarks. Super Pages show their true potential in high-memory applications like PS2 emulation.

Having seen the performance gains Super Pages can bring to the Raspberry Pi, let’s now dive into the technical aspects of the feature.

Implementing Super Pages

The first challenge was figuring out how to allocate a contiguous block of memory using shmem. The Shared Memory Virtual Filesystem (shmem) is used as a flexible memory mechanism that allows the GPU and CPU to share access to BOs through the system’s temporary filesystem, tmpfs. tmpfs is a volatile filesystem that stores files in RAM, making it ideal for temporary or high-speed data that doesn’t need to persist on RAM.

For example, to allocate a 256KB BO across four 64KB pages, we need four contiguous 64KB memory blocks. However, by default, tmpfs only allocates memory in PAGE_SIZE chunks (as seen in shmem_file_setup()), whereas PAGE_SIZE is 4KB on the Raspberry Pi 4 and 16KB on the Raspberry Pi 5. Since the function drm_gem_object_init() — which initializes an allocated shmem-backed GEM object — relies on shmem_file_setup() to back these objects in memory, we had to consider alternatives, as the default PAGE_SIZE would divide memory into increments that are too small to ensure the large, contiguous blocks needed by the GPU.

The solution we proposed was to create drm_gem_object_init_with_mnt(), which allows us to specify the tmpfs mountpoint where the GEM object will be created. This enables us to allocate our BOs in a mountpoint that supports larger page sizes. Additionally, to ensure that our BOs are allocated in the correct mountpoint, we introduced drm_gem_shmem_create_with_mnt(), which allows the mountpoint to be specified when creating a new DRM GEM shmem object.

[PATCH v6 04/11] drm/gem: Create a drm_gem_object_init_with_mnt() function

[PATCH v6 06/11] drm/gem: Create shmem GEM object in a given mountpoint

The next challenge was figuring out how to create a new mountpoint that would allow for different page sizes based on the allocation. Simply creating a new tmpfs mountpoint with a fixed bigger page size wouldn’t suffice, as we needed flexibility for various allocations. Inspired by the i915 driver, we decided to use a tmpfs mountpoint with the “huge=within_size” flag. This flag, which requires the kernel to be configured with CONFIG_TRANSPARENT_HUGEPAGE, enables the allocation of huge pages.

Transparent Huge Pages (THP) is a kernel feature that automatically manages large memory pages to improve performance without needing changes from applications. THP dynamically combines smaller pages into larger ones, typically 2MB, reducing memory management overhead and improving cache efficiency.

To support our new allocation strategy, we created a dedicated tmpfs mountpoint for V3D, called gemfs, which provides us an ideal space for managing these larger allocations.

[PATCH v6 05/11] drm/v3d: Introduce gemfs

With everything in place for contiguous allocations, the next step was configuring V3D to enable Big/Super Page support.

We began by addressing a major source of memory pressure on the Raspberry Pi: the current 128KB alignment for allocations in the virtual memory space. This alignment wastes space when handling small BO allocations, especially since the userspace driver performs a large number of these small allocations.

As a result, we can’t fully utilize the 4GB address space available for the GPU on the Raspberry Pi 4 or 5. For example, we can currently allocate up to 32,000 BOs of 4KB (~140MB) and 3,000 BOs of 400KB (~1.3GB). This becomes a limitation for memory-intensive applications. By reducing the page alignment to 4KB, we can significantly increase the number of BOs, allowing up to 1,000,000 BOs of 4KB (~4GB) and 10,000 BOs of 400KB (~4GB).

Therefore, the first change I made was reducing the VA alignment of all allocations to 4KB.

[PATCH v6 07/11] drm/v3d: Reduce the alignment of the node allocation

With the alignment issue resolved, we can now implement the code to properly set the flags on the Page Table Entries (PTE) for Big/Super Pages. Setting these flags is straightforward — a simple bitwise operation. The challenge lies in determining which BOs can be allocated in Super Pages. For a BO to be eligible for a Big Page, its virtual address must be aligned to 64KB, and the same applies to its physical address. Same thing for Super Pages, but now the addresses must be aligned to 1MB.

If the BO qualifies for a Big/Super Page, we need to iterate over 16 4KB pages (for Big Pages) or 256 4KB pages (for Super Pages) and insert the appropriate PTE.

Additionally, we modified the way we iterate through the BO’s memory. This was necessary because the THP may not always allocate the entire BO contiguously. For example, it might only allocate contiguously 1MB of a 2MB block. To handle this, we now iterate over the blocks of contiguous memory scattered across the scatterlist, ensuring that each segment is properly handled during the allocation process.

What is a scatterlist? It is a Linux Kernel data structure that manages non-contiguous memory as if it were contiguous. It organizes separate memory blocks into a single logical buffer, allowing efficient data handling, especially in Direct Memory Access (DMA) operations, without needing a physically contiguous memory allocation.

[PATCH v6 08/11] drm/v3d: Support Big/Super Pages when writing out PTEs

However, the last few patches alone don’t fully enable the use of Super Pages. While PATCH 08/11 technically allows for Super Pages, we’re still relying on DRM GEM shmem objects, meaning allocations are still happening in PAGE_SIZE chunks. Although Big/Super Pages could potentially be used if the system naturally allocated 1MB or 64KB contiguously, this is quite rare and not our intended outcome. Our goal is to actively use Big/Super Pages as much as possible.

To achieve this, we’ll utilize the V3D-specific mountpoint we created earlier for BO allocation whenever possible. By creating BOs through drm_gem_shmem_create_with_mnt(), we can ensure that large pages are allocated contiguously when possible, enabling the consistent use of Big/Super Pages.

[PATCH v6 09/11] drm/v3d: Use gemfs/THP in BO creation if available

And there you have it — Big/Super Pages are now fully enabled in V3D. The only requirement to activate this feature in any given kernel is ensuring that CONFIG_TRANSPARENT_HUGEPAGE is enabled.

Final Words

You can learn more about ongoing enhancements to the Raspberry Pi driver stack in this XDC 2024 talk by José María “Chema” Casanova Crespo. In the talk, Chema discusses the Super Pages work I developed, along with other advancements in the driver stack.

Of course, there are still plenty of improvements on the horizon at Igalia. I’m currently experimenting with 64KB CLE allocations in user-space, and I hope to share more good news soon.

Finally, I’d like to express my gratitude to Iago Toral and Tvrtko Ursulin for their invaluable support in developing Super Pages for the V3D kernel driver. Thank you both for sharing your experience with me!

Linux 6.8: AMD HDR and Raspberry Pi 5

Tue, 02 Apr 2024 11:00:00 +0000

The Linux kernel 6.8 came out on March 10th, 2024, bringing brand-new features and plenty of performance improvements on different subsystems. As part of Igalia, I’m happy to be an active part of many features that are released in this version, and today I’m going to review some of them.

Linux 6.8 is packed with a lot of great features, performance optimizations, and new hardware support. In this release, we can check the Intel Xe DRM driver experimentally, further support for AMD Zen 5 and other upcoming AMD hardware, initial support for the Qualcomm Snapdragon 8 Gen 3 SoC, the Imagination PowerVR DRM kernel driver, support for the Nintendo NSO controllers, and much more.

Igalia is widely known for its contributions to Web Platforms, Chromium, and Mesa. But, we also make significant contributions to the Linux kernel. This release shows some of the great work that Igalia is putting into the kernel and strengthens our desire to keep working with this great community.

Let’s take a deep dive into Igalia’s major contributions to the 6.8 release:

AMD HDR & Color Management

You may have seen the release of a new Steam Deck last year, the Steam Deck OLED. What you may not know is that Igalia helped bring this product to life by putting some effort into the AMD driver-specific color management properties implementation. Melissa Wen, together with Joshua Ashton (Valve), and Harry Wentland (AMD), implemented several driver-specific properties to allow Gamescope to manage color features provided by the AMD hardware to fit HDR content and improve gamers’ experience.

She has explained all features implemented in the AMD display kernel driver in two blog posts and a 2023 XDC talk:

Async Flip

André Almeida worked together with Simon Ser (SourceHut) to provide support for asynchronous page-flips in the atomic API. This feature targets users who want to present a new frame immediately, even if after missing a V-blank. This feature is particularly useful for applications with high frame rates, such as gaming.

Raspberry Pi 5

Raspberry Pi 5 was officially released on October 2023 and Igalia was ready to bring top-notch graphics support for it. Although we still can’t use the RPi 5 with the mainline kernel, it is superb to see some pieces coming upstream. Iago Toral worked on implementing all the kernel support needed for the V3D 7.1.x driver.

With the kernel patches, by the time the RPi 5 was released, it already included a fully 3.1 OpenGL ES and Vulkan 1.2 compliant driver implemented by Igalia.

GPU stats and CPU jobs for the Raspberry Pi 4/5

Apart from the release of the Raspberry Pi 5, Igalia is still working on improving the whole Raspberry Pi environment. I worked, together with José Maria “Chema” Casanova, implementing the support for GPU stats on the V3D driver. This means that RPi 4/5 users now can access the usage percentage of the GPU and they can access the statistics by process or globally.

I also worked, together with Melissa, implementing CPU jobs for the V3D driver. As the Broadcom GPU isn’t capable of performing some operations, the Vulkan driver uses the CPU to compensate for it. In order to avoid stalls in the job submission, now CPU jobs are part of the kernel and can be easily synchronized though with synchronization objects.

If you are curious about the CPU job implementation, you can check this blog post.

Other Contributions & Fixes

Sometimes we don’t contribute to a major feature in the release, however we can help improving documentation and sending fixes. André also contributed to this release by documenting the different AMD GPU reset methods, making it easier to understand by future users.

During Igalia’s efforts to improve the general users’ experience on the Steam Deck, Guilherme G. Piccoli noticed a message in the kernel log and readily provided a fix for this PCI issue.

Outside of the Steam Deck world, we can check some of Igalia’s work on the Qualcomm Adreno GPUs. Although most of our Adreno-related work is located at the user-space, Danylo Piliaiev sent a couple of kernel fixes to the msm driver, fixing some hangs and some CTS tests.

We also had contributions from our 2023 Igalia CE student, Nia Espera. Nia’s project was related to mobile Linux and she managed to write a couple of patches to the kernel in order to add support for the OnePlus 9 and OnePlus 9 Pro devices.

If you are a student interested in open-source and would like to have a first exposure to the professional world, check if we have openings for the Igalia Coding Experience. I was a CE student myself and being mentored by a Igalian was a incredible experience.

Check the complete list of Igalia’s contributions for the 6.8 release

Authored (57):

André Almeida (2)

Danylo Piliaiev (2)

Guilherme G. Piccoli (1)

PCI: Only override AMD USB controller if required

Iago Toral Quiroga (4)

Maíra Canal (17)

Melissa Wen (27)

Nia Espera (4)

Signed-off-by (88):

André Almeida (4)

Danylo Piliaiev (2)

Guilherme G. Piccoli (1)

PCI: Only override AMD USB controller if required

Iago Toral Quiroga (4)

Jose Maria Casanova Crespo (2)

Maíra Canal (28)

Melissa Wen (43)

Nia Espera (4)

Acked-by (4):

Jose Maria Casanova Crespo (2)

Maíra Canal (1)

MAINTAINERS: Drop Emma Anholt from all M lines.

Melissa Wen (1)

MAINTAINERS: Add Maira to V3D maintainers

Reviewed-by (30):

André Almeida (1)

drm: introduce DRM_CAP_ATOMIC_ASYNC_PAGE_FLIP

Christian Gmeiner (1)

drm/etnaviv: Convert to platform remove callback returning void

Iago Toral Quiroga (20)

Maíra Canal (4)

Melissa Wen (4)

Tested-by (1):

Guilherme G. Piccoli (1)

pstore/ram: Fix crash when setting number of cpus to an odd number

Introducing CPU jobs to the Raspberry Pi

Thu, 11 Jan 2024 13:30:00 +0000

Igalia is always working hard to improve 3D rendering drivers of the Broadcom VideoCore GPU, found in Raspberry Pi devices. One of our most recent efforts in this sense was the implementation of CPU jobs from the Vulkan driver to the V3D kernel driver.

What are CPU jobs and why do we need them?

In the V3DV driver, there are some Vulkan commands that cannot be performed by the GPU alone, so we implement those as CPU jobs on Mesa. A CPU job is a job that requires CPU intervention to be performed. For example, in the Broadcom VideoCore GPUs, we don’t have a way to calculate the timestamp. But we need the timestamp for Vulkan timestamp queries. Therefore, we need to calculate the timestamp on the CPU.

A CPU job in userspace also implies CPU stalling. Sometimes, we need to hold part of the command submission flow in order to correctly synchronize their execution. This waiting period caused the CPU to stall, thereby preventing the continuous submission of jobs to the GPU. To mitigate this issue, we decided to move CPU job mechanisms from the V3DV driver to the V3D kernel driver.

In the V3D kernel driver, we have different kinds of jobs: RENDER jobs, BIN jobs, CSD jobs, TFU jobs, and CLEAN CACHE jobs. For each of those jobs, we have a DRM scheduler instance that helps us to synchronize the jobs.

If you want to know more about the different kinds of V3D jobs, check out this November Update: Exploring V3D blogpost, where I explain more about all the V3D IOCTLs and jobs.

Jobs of the same kind are submitted, dispatched, and processed in the same order they are executed, using a standard first-in-first-out (FIFO) queue system. We can synchronize different jobs across different queues using DRM syncobjs. More about the V3D synchronization framework and user extensions can be learned in this two-part blog post from Melissa Wen.

From the kernel documentation, a DRM syncobj (synchronisation objects) are containers for stuff that helps sync up GPU commands. They’re super handy because you can use them in your own programs, share them with other programs, and even use them across different DRM drivers. Mostly, they’re used for making Vulkan fences and semaphores work.

By moving the CPU job from userspace to the kernel, we can make use of the DRM schedule queues and all the advantages it brings with it. For this, we created a new type of job in the V3D kernel driver, a CPU job, which also means creating a new DRM scheduler instance and a CPU job queue. Now, instead of stalling the submission thread waiting for the GPU to idle, we can use DRM syncobjs to synchronize both CPU and GPU jobs in a submission, providing more efficient usage of the GPU.

How did we implement the CPU jobs in the kernel driver?

After we decided to have a CPU job implementation in the kernel space, we could think about two possible implementations for this job: creating an IOCTL for each type of CPU job or using a user extension to provide a polymorphic behavior to a single CPU job IOCTL.

We have different types of CPU jobs (indirect CSD jobs, timestamp query jobs, copy query results jobs…) and each of them has a common infrastructure of allocation and synchronization but performs different operations. Therefore, we decided to go with the option to use user extensions.

On Melissa’s blogpost, she digs deep into the implementation of generic IOCTL extensions in the V3D kernel driver. But, to put it simply, instead of expanding the data struct for each IOCTL every time we need to add a new feature, we define a user extension chain instead. As we add new optional interfaces to control the IOCTL, we define a new extension struct that can be linked to the IOCTL data only when required by the user.

Therefore, we created a new IOCTL, drm_v3d_submit_cpu, which is used to submit any type of CPU job. This single IOCTL can be extended by a user extension, which allows us to reuse the common infrastructure - avoiding code repetition - and yet use the user extension ID to identify the type of job and depending on the type of job, perform a certain operation.

struct drm_v3d_submit_cpu {
        /* Pointer to a u32 array of the BOs that are referenced by the job.
         *
         * For DRM_V3D_EXT_ID_CPU_INDIRECT_CSD, it must contain only one BO,
         * that contains the workgroup counts.
         *
         * For DRM_V3D_EXT_ID_TIMESTAMP_QUERY, it must contain only one BO,
         * that will contain the timestamp.
         *
         * For DRM_V3D_EXT_ID_CPU_RESET_TIMESTAMP_QUERY, it must contain only
         * one BO, that contains the timestamp.
         *
         * For DRM_V3D_EXT_ID_CPU_COPY_TIMESTAMP_QUERY, it must contain two
         * BOs. The first is the BO where the timestamp queries will be written
         * to. The second is the BO that contains the timestamp.
         *
         * For DRM_V3D_EXT_ID_CPU_RESET_PERFORMANCE_QUERY, it must contain no
         * BOs.
         *
         * For DRM_V3D_EXT_ID_CPU_COPY_PERFORMANCE_QUERY, it must contain one
         * BO, where the performance queries will be written.
         */
        __u64 bo_handles;

        /* Number of BO handles passed in (size is that times 4). */
        __u32 bo_handle_count;

        __u32 flags;

        /* Pointer to an array of ioctl extensions*/
        __u64 extensions;
};

Now, we can create a CPU job and submit it with a CPU job user extension.

And which extensions are available?

DRM_V3D_EXT_ID_CPU_INDIRECT_CSD: this CPU job allows us to submit an indirect CSD job. An indirect CSD job is a job that, when executed in the queue, will map an indirect buffer, read the dispatch parameters, and submit a regular dispatch. This CPU job is used in Vulkan calls like vkCmdDispatchIndirect().
DRM_V3D_EXT_ID_CPU_TIMESTAMP_QUERY: this CPU job calculates the query timestamp and updates the query availability by signaling a syncobj. This CPU job is used in Vulkan calls like vkCmdWriteTimestamp().
DRM_V3D_EXT_ID_CPU_RESET_TIMESTAMP_QUERY: this CPU job resets the timestamp queries based on the value offset of the first query. This CPU job is used in Vulkan calls like vkCmdResetQueryPool() for timestamp queries.
DRM_V3D_EXT_ID_CPU_COPY_TIMESTAMP_QUERY: this CPU job copies the complete or partial result of a query to a buffer. This CPU job is used in Vulkan calls like vkCmdCopyQueryPoolResults() for timestamp queries.
DRM_V3D_EXT_ID_CPU_RESET_PERFORMANCE_QUERY: this CPU job resets the performance queries by resetting the values of the perfmons. This CPU job is used in Vulkan calls like vkCmdResetQueryPool() for performance queries.
DRM_V3D_EXT_ID_CPU_COPY_PERFORMANCE_QUERY: similar to DRM_V3D_EXT_ID_CPU_COPY_TIMESTAMP_QUERY, this CPU job copies the complete or partial result of a query to a buffer. This CPU job is used in Vulkan calls like vkCmdCopyQueryPoolResults() for performance queries.

The CPU job IOCTL structure is similar to any other V3D job. We allocate the job struct, parse all the extensions, init the job, look up the BOs and lock its reservations, add the proper dependencies, and push the job to the DRM scheduler entity.

When running a CPU job, we execute the following code:

static const v3d_cpu_job_fn cpu_job_function[] = {
        [V3D_CPU_JOB_TYPE_INDIRECT_CSD] = v3d_rewrite_csd_job_wg_counts_from_indirect,
        [V3D_CPU_JOB_TYPE_TIMESTAMP_QUERY] = v3d_timestamp_query,
        [V3D_CPU_JOB_TYPE_RESET_TIMESTAMP_QUERY] = v3d_reset_timestamp_queries,
        [V3D_CPU_JOB_TYPE_COPY_TIMESTAMP_QUERY] = v3d_copy_query_results,
        [V3D_CPU_JOB_TYPE_RESET_PERFORMANCE_QUERY] = v3d_reset_performance_queries,
        [V3D_CPU_JOB_TYPE_COPY_PERFORMANCE_QUERY] = v3d_copy_performance_query,
};

static struct dma_fence *
v3d_cpu_job_run(struct drm_sched_job *sched_job)
{
        struct v3d_cpu_job *job = to_cpu_job(sched_job);
        struct v3d_dev *v3d = job->base.v3d;

        v3d->cpu_job = job;

        if (job->job_type >= ARRAY_SIZE(cpu_job_function)) {
                DRM_DEBUG_DRIVER("Unknown CPU job: %d\n", job->job_type);
                return NULL;
        }

        trace_v3d_cpu_job_begin(&v3d->drm, job->job_type);

        cpu_job_function[job->job_type](job);

        trace_v3d_cpu_job_end(&v3d->drm, job->job_type);

        return NULL;
}

The interesting thing is that each CPU job type executes a completely different operation.

The complete kernel implementation has already landed in drm-misc-next and can be seen right here.

What did we change in Mesa-V3DV to use the new kernel-V3D CPU job?

After landing the kernel implementation, I needed to accommodate the new CPU job approach in the userspace.

A fundamental rule is not to cause regressions, i.e., to keep backwards userspace compatibility with old versions of the Linux kernel. This means we cannot break new versions of Mesa running in old kernels. Therefore, we needed to create two paths: one preserving the old way to perform CPU jobs and the other using the kernel to perform CPU jobs.

So, for example, the indirect CSD job used to add two different jobs to the queue: a CPU job and a CSD job. Now, if we have the CPU job capability in the kernel, we only add a CPU job and the CSD job is dispatched from within the kernel.

-   list_addtail(&csd_job->list_link, &cmd_buffer->jobs);
+
+   /* If we have a CPU queue we submit the CPU job directly to the
+    * queue and the CSD job will be dispatched from within the kernel
+    * queue, otherwise we will have to dispatch the CSD job manually
+    * right after the CPU job by adding it to the list of jobs in the
+    * command buffer.
+    */
+   if (!cmd_buffer->device->pdevice->caps.cpu_queue)
+      list_addtail(&csd_job->list_link, &cmd_buffer->jobs);

Furthermore, now we can use syncobjs to sync the CPU jobs. For example, in the timestamp query CPU job, we used to stall the submission thread and wait for completion of all work queued before the timestamp query. Now, we can just add a barrier to the CPU job and it will be properly synchronized by the syncobjs without stalling the submission thread.

   /* The CPU job should be serialized so it only executes after all previously
    * submitted work has completed
    */
   job->serialize = V3DV_BARRIER_ALL;

We were able to test the implementation using multiple CTS tests, such as dEQP-VK.compute.pipeline.indirect_dispatch.*, dEQP-VK.pipeline.monolithic.timestamp.*, dEQP-VK.synchronization.*, dEQP-VK.query_pool.* and dEQP-VK.multiview.*.

The userspace implementation has already landed in Mesa and the full implementation can be checked in this MR.

More about the on-going challenges in the Raspberry Pi driver stack can be checked during this XDC 2023 talk presented by Iago Toral, Juan Suárez and myself. During this talk, Iago mentioned the CPU job work that we have been doing.

Also I cannot finish this post without thanking Melissa Wen and Iago Toral for all the help while developing the CPU jobs for the V3D kernel driver.

May Update: Finishing my Second Igalia CE

Mon, 22 May 2023 12:30:00 +0000

After finishing up my first Igalia Coding Experience in January, I got the amazing opportunity to keep working in the DRI community by extending my Igalia CE to a second round. Huge thanks to Igalia for providing me with this opportunity!

Another four months passed by and here I am completing another milestone with Igalia. Previously, in the last final reports, I described GSoC as “an experience to get a better understanding of what open source is” and the first round of the Igalia CE as “an opportunity for me to mature my knowledge of technical concepts”. My second round of the Igalia CE was a period for broadening my horizons.

I had the opportunity to deepen my knowledge of a new programming language and learn more about Kernel Mode Setting (KMS). I took my time learning more about Vulkan and the Linux graphics stack. All of this new knowledge about the DRM infrastructure fascinated me and made me excited to keep developing.

So, this is a summary report of my journey at my second Igalia CE.

Wrapping Up

First, I took some time to wrap up the contributions of my previous Igalia CE. In my January Update, I described the journey to include IGT tests for V3D. But at the time, I hadn’t yet sent the final versions of the tests. Right when I started my second Igalia CE, I sent the final versions of the V3D tests, which were accepted and merged.

Series	Status
[PATCH i-g-t 0/6] V3D Job Submission Tests	Accepted
[PATCH i-g-t 0/3] V3D Mixed Job Submission Tests	Accepted

Rustgem

The first part of my Igalia CE was focused on rewriting the VGEM driver in Rust. VGEM (Virtual GEM Provider) is a minimal non-hardware-backed GEM (Graphics Execution Manager) service. It is used with non-native 3D hardware for buffer sharing between the X server and DRI.

The goal of the project was to explore Rust in the DRM subsystem and have a working VGEM driver written in Rust. Rust is a blazingly fast and memory-efficient language with its powerful ownership model. It was really exciting to learn more about Rust and implement from the beginning a DRM driver.

During the project, I wrote two blog posts describing the technical aspects of rustgem driver. If you are interested in this project, check them out!

Date	Blogpost
28th February	Rust for VGEM
22th March	Adding a Timeout feature to Rustgem

By the end of the first half of the Igalia CE, I sent an RFC patch with the rustgem driver. Thanks to Asahi Lina, the Rust for Linux folks, and Daniel Vetter for all the feedback provided during the development of the driver. I still need to address some feedback and rebase the series on top of the new pin-init API, but I hope to see this driver upstream soon. You can check the driver’s current status in this PR.

Series	Status
[RFC PATCH 0/9] Rust version of the VGEM driver	In Review

Apart from rewriting the VGEM driver, I also sent a couple of improvements to the C version of the VGEM driver and its IGT tests. I found a missing mutex_destroy on the code and also an unused struct.

Patches	Status
[PATCH] drm/vgem: add missing mutex_destroy	Accepted
[PATCH] drm/vgem: Drop struct drm_vgem_gem_object	Accepted

On the IGT side, I added some new tests to the VGEM tests. I wanted to ensure that my driver returned the correct values for all possible error paths, so I wrote this IGT test. Initially, it was just for me, but I decided to submit it upstream.

Series	Status
[PATCH v3 i-g-t 0/2] Add negative tests to VGEM	Accepted

Virtual Kernel Mode Setting (VKMS)

Focusing on the VKMS was the major goal of the second part of my Igalia CE. Melissa Wen is one of the maintainers of the VKMS, and she provided me with a fantastic opportunity to learn more about the VKMS. So far, I haven’t dealt with displays, and learning new concepts in the graphics stack was great.

Rotating Planes

VKMS is a software-only KMS driver that is quite useful for testing and running X (or similar compositors) on headless machines. At the time, the driver didn’t have any support for optional plane properties, such as rotation and blend mode. Therefore, my goal was to implement the first plane property of the driver: rotation. I described the technicalities of this challenge in this blog post, but I can say that it was a nice challenge of this mentorship project.

In the end, we have the first plane property implemented for the VKMS and it is already committed. Together with the VKMS part, I sent a series to the IGT mailing list with some improvements to the kms_rotation_crc tests. These improvements included adding new tests for rotation with offset and reflection and the isolation of some Intel-specific tests.

Series	Status
[PATCH v4 0/6] drm/vkms: introduce plane rotation property	Accepted
[PATCH 1/2] drm/vkms: Add kernel-doc to the function vkms_compose_row()	In Review
[PATCH i-g-t 0/4] kms_rotation_crc improvements and generalization	In Review

Improvements

As I was working with the rotation series, I discovered a couple of things that could be improved in the VKMS driver. Last year, Igor Torrente sent a series to VKMS that changed the composition work in the driver. Before his series, the plane composition was executed on top of the primary plane. Now, the plane composition is executed on top of the CRTC.

Although his series was merged, some parts of the code still considered that the composition was executed on top of the primary plane, limiting the VKMS capabilities. So I sent a couple of patches to the mailing list, improving the handling of the primary plane and allowing full alpha blending on all planes.

Moreover, I sent a series that added a module parameter to set a background color to the CRTC. This work raised an interesting discussion about the need for this property by the user space and whether this parameter should be a KMS property.

Apart from introducing the rotation property to the VKMS driver, I also took my time to implement two other properties: alpha and blend mode. This series is still awaiting review, but it would be a nice addition to the VKMS, increasing its IGT test coverage rate.

Finally, I found a bug in the RGB565 conversion. The RGB565 conversion to ARGB16161616 involves some fixed-point operations and, when running the pixel-format IGT test, I verified that the RGB565 test was failing. So, some of those fixed-point operations were returning erroneous values. I checked that the RGB coefficients weren’t being rounded when converted from fixed-point to integers. But, this should happen in order to provided the proper coefficient values. Therefore, the fix was: implement a new helper that rounds the fixed-point value when converting it to a integer.

After performing all this work on the VKMS, I sent a patch adding myself as a VKMS maintainer, which was acked by Javier Martinez and Melissa Wen. So now, I’m working together with Melissa, Rodrigo Siqueira and all DRI community to improve and maintain the VKMS driver.

Series	Status
[PATCH v2 0/2] Update the handling of the primary plane	Accepted
[PATCH v2 1/2] drm/vkms: allow full alpha blending on all planes	Accepted
[PATCH v2] drm/vkms: add module parameter to set background color	In Review
[PATCH] drm/vkms: Implement all blend mode properties	In Review
[PATCH v3 1/2] drm: Add fixed-point helper to get rounded integer values	Accepted

Virtual Hardware

A couple of years ago, Sumera Priyadarsini, an Outreachy intern, worked on a Virtual Hardware functionality for the VKMS. The idea was to add a Virtual Hardware or vblank-less mode as a kernel parameter to enable VKMS to emulate virtual devices. This means no vertical blanking events occur and page flips are completed arbitrarily when required for updating the frame. Unfortunately, she wasn’t able to wrap things up and this ended up never being merged into VKMS.

Melissa suggested rebasing this series and now we can have the Virtual Hardware functionality working on the current VKMS. This was a great work by Sumera and my work here was just to adapt her changes to the new VKMS code.

Series	Status
[PATCH 0/2] drm/vkms: Enable Virtual Hardware support	In Review

Bug Fixing!

Finally, I was in the last week of the project, just wrapping things up, when I decided to run the VKMS CI. I had recently committed the rotation series and I had run the CI before, but to my surprise, I got the following output:

[root@fedora igt-gpu-tools]# ./build/tests/kms_writeback
IGT-Version: 1.27.1-gce51f539 (x86_64) (Linux: 6.3.0-rc4-01641-gb8e392245105-dirty x86_64)
(kms_writeback:1590) igt_kms-WARNING: Output Writeback-1 could not be assigned to a pipe
Starting subtest: writeback-pixel-formats
Subtest writeback-pixel-formats: SUCCESS (0.000s)
Starting subtest: writeback-invalid-parameters
Subtest writeback-invalid-parameters: SUCCESS (0.001s)
Starting subtest: writeback-fb-id
Subtest writeback-fb-id: SUCCESS (0.020s)
Starting subtest: writeback-check-output
(kms_writeback:1590) CRITICAL: Test assertion failure function get_and_wait_out_fence, file ../tests/kms_writeback.c:288:
(kms_writeback:1590) CRITICAL: Failed assertion: ret == 0
(kms_writeback:1590) CRITICAL: Last errno: 38, Function not implemented
(kms_writeback:1590) CRITICAL: sync_fence_wait failed: Timer expired
Stack trace:
  #0 ../lib/igt_core.c:1963 __igt_fail_assert()
  #1 [get_and_wait_out_fence+0x83]
  #2 ../tests/kms_writeback.c:337 writeback_sequence()
  #3 ../tests/kms_writeback.c:360 __igt_unique____real_main481()
  #4 ../tests/kms_writeback.c:481 main()
  #5 ../sysdeps/nptl/libc_start_call_main.h:74 __libc_start_call_main()
  #6 ../csu/libc-start.c:128 __libc_start_main@@GLIBC_2.34()
  #7 [_start+0x25]
Subtest writeback-check-output failed.
**** DEBUG ****
(kms_writeback:1590) CRITICAL: Test assertion failure function get_and_wait_out_fence, file ../tests/kms_writeback.c:288:
(kms_writeback:1590) CRITICAL: Failed assertion: ret == 0
(kms_writeback:1590) CRITICAL: Last errno: 38, Function not implemented
(kms_writeback:1590) CRITICAL: sync_fence_wait failed: Timer expired
(kms_writeback:1590) igt_core-INFO: Stack trace:
(kms_writeback:1590) igt_core-INFO:   #0 ../lib/igt_core.c:1963 __igt_fail_assert()
(kms_writeback:1590) igt_core-INFO:   #1 [get_and_wait_out_fence+0x83]
(kms_writeback:1590) igt_core-INFO:   #2 ../tests/kms_writeback.c:337 writeback_sequence()
(kms_writeback:1590) igt_core-INFO:   #3 ../tests/kms_writeback.c:360 __igt_unique____real_main481()
(kms_writeback:1590) igt_core-INFO:   #4 ../tests/kms_writeback.c:481 main()
(kms_writeback:1590) igt_core-INFO:   #5 ../sysdeps/nptl/libc_start_call_main.h:74 __libc_start_call_main()
(kms_writeback:1590) igt_core-INFO:   #6 ../csu/libc-start.c:128 __libc_start_main@@GLIBC_2.34()
(kms_writeback:1590) igt_core-INFO:   #7 [_start+0x25]
****  END  ****
Subtest writeback-check-output: FAIL (1.047s)

🫠 🫠 🫠 🫠 🫠 🫠 🫠 🫠 🫠 🫠 🫠 🫠 🫠 🫠 🫠 🫠

Initially, I thought I had introduced the bug with my rotation series. Turns out, I just had made it more likely to happen. This bug has been hidden in VKMS for a while, but it happened just on rare occasions. Yeah, I’m talking about a race condition… The kind of bug that just stays hidden in your code for a long while.

When I started to debug, I thought it was a performance issue. But then, I increased the timeout to 10 seconds and even then the job wouldn’t finish. So, I thought that it could be a deadlock. But after inspecting the DRM internal locks and the VKMS locks, it didn’t seem the case.

Melissa pointed me to a hint: there was one framebuffer being leaked when removing the driver. I discovered that it was the writeback framebuffer. It meant that the writeback job was being queued, but it wasn’t being signaled. So, the problem was inside the VKMS locking mechanism.

After tons of GDB and ftrace, I was able to find out that the composer was being set twice without any calls to the composer worker. I changed the internal locks a bit and I was able to run the test repeatedly for minutes! I sent the fix for review and now I’m just waiting for a Reviewed-by.

Patches	Status
[PATCH] drm/vkms: Fix race-condition between the hrtimer and the atomic commit	In Review

While debugging, I found some things that could be improved in the VKMS writeback file. So, I decided to send a series with some minor improvements to the code.

Series	Status
[PATCH 0/3] drm/vkms: Minor Improvements	In Review

Improving IGT tests

If you run all IGT KMS tests on the VKMS driver, you will see that some tests will fail. That’s not what we would expect: we would expect that all tests would pass or skip. The tests could fail due to errors in the VKMS driver or be wrong exceptions on the IGT side. So, on the final part of my Igalia CE, I inspected a couple of IGT failures and sent fixes to address the errors.

Linux Kernel

This patch is a revival of a series I sent in January to fix the IGT test kms_addfb_basic@addfb25-bad-modifier. This test also failed in VC4, and I investigated the reason in January. I sent a patch to guarantee that the test would pass and after some feedback, I came down to a dead end. So, I left this patch aside for a while and decided to recapture it now. Now, with this patch being merged, we can guarantee that the test kms_addfb_basic@addfb25-bad-modifier is passing for multiple drivers.

Patches	Status
[PATCH] drm/gem: Check for valid formats	Accepted

IGT

On the IGT side, I sent a couple of improvements to the tests. The failure was usually just a scenario that the test didn’t consider. For example, the kms_plane_scaling test was failing in VKMS, because it didn’t consider the case in which the driver did not have the rotation property. As VKMS didn’t use to have the rotation property, the tests were failing instead of skipping. Therefore, I just developed a path for drivers without the rotation property for the tests to skip.

I sent improvements to the kms_plane_scaling, kms_flip, and kms_plane tests, making the tests pass or skip on all cases for the VKMS.

Patches	Status
[PATCH i-g-t] tests/kms_plane_scaling: negative tests can return -EINVAL or -ERANGE	Accepted
[PATCH i-g-t] tests/kms_plane_scaling: fix variable misspelling	Accepted
[PATCH i-g-t] tests/kms_plane_scaling: remove unused parameters	Accepted
[PATCH i-g-t] tests/kms_plane_scaling: Only set rotation if rotation != rotate-0	Accepted
[PATCH v2 i-g-t] tests/kms_flip: Check if is Intel device before doing all the setup	Accepted
[PATCH i-g-t v2] tests/kms_plane: allow pixel-format tests to run on drivers without legacy LUT	In Review

VKMS CI List

One important thing to VKMS is creating a baseline of generic KMS tests that should pass. This way, we can test new contributions against this baseline and avoid introducing regressions in the codebase. I sent a patch to IGT to create a testlist for the VKMS driver with all the KMS tests that must pass on the VKMS driver. This is great for maintenance, as we can run the testlist to ensure that the VKMS functionalities are preserved.

With new features being introduced in VKMS, it is important to keep the test list updated. So, I verified the test results and updated this test list during my time at the Igalia CE. I intend to keep this list updated as long as I can.

Series	Status
[PATCH i-g-t] tests/vkms: Create a testlist to the vkms DRM driver	Accepted
[PATCH i-g-t 0/3] tests/vkms: Update VKMS’s testlist	Accepted

Acknowledgment

First, I would like to thank my great mentor Melissa Wen. Melissa and I are completing a year together as mentee and mentor and it has been an amazing journey. Since GSoC, Melissa has been helping me by answering every single question I have and providing me with great encouragement. I have a lot of admiration for her and I’m really grateful for having her as my mentor during the last year.

Also, I would like to thank Igalia for giving me this opportunity to keep working in the DRI community and learning more about this fascinating topic. Thanks to all Igalians that helped through this journey!

Moreover, I would like to thank the DRI community for reviewing my patches and giving me constructive feedback. Especially, I would like to thank Asahi Lina, Daniel Vetter and the Rust for Linux folks for all the help with the rustgem driver.

Cross-Compiling CTS for the Raspberry Pi 4

Tue, 16 May 2023 12:00:00 +0000

This blogpost was actually written partially in November/December 2022 while I was developing IGT tests for the V3D driver. I ended up leaving it aside for a while and now, I came back and finished the last loose ends. That’s why I’m referencing the time where I was fighting against V3D’s noop jobs.

Currently, during my Igalia Coding Experience, I’m working on the V3D’s IGT tests and therefore, I’m dealing a lot with the Raspberry Pi 4. During the project, I had a real struggle to design the tests for the v3d_submit_cl ioctl, as I was not capable of submit a proper noop job to the GPU.

In order to debug the tests, my mentor Melissa Wen suggested to me to run the CTS tests to reproduce a noop job and debug it through Mesa. I cloned the CTS repository into my Raspberry Pi 4 and I tried to compile, but my Raspberry Pi 4 went OOM. This sent me on a journey to cross-compile CTS for the Raspberry Pi 4. I decided to compile this journey into this blogpost.

During this blogpost, I’m using a Raspbian OS with desktop 64-bit.

Installing Mesa

First, you need to install Mesa on the Raspberry Pi 4. I decided to compile Mesa on the Raspberry Pi 4 itself, but maybe one day, I can write a blogpost about cross-compiling Mesa.

1. Installing libdrm

Currently, the Raspbian repositories only provide libdrm 2.4.104 and Mesa’s main branch needs libdrm >=2.4.109. So, first, let’s install libdrm 2.4.109 on the Raspberry Pi 4.

First, let’s make sure that you have meson installed on your RPi4. We will need meson to build libdrm and Mesa. I’m installing meson through pip3 because we need a meson version greater than 0.60 to build Mesa.

# On the Raspberry Pi 4
$ sudo pip3 install meson

Then, you can install libdrm 2.4.109 on the RPi4.

# On the Raspberry Pi 4
$ wget https://dri.freedesktop.org/libdrm/libdrm-2.4.114.tar.xz
$ tar xvpf libdrm-2.4.114.tar.xz
$ cd libdrm-2.4.114
$ mkdir build
$ cd build
$ FLAGS="-O2 -march=armv8-a+crc+simd -mtune=cortex-a72" \
    CXXFLAGS="-O2 -march=armv8-a+crc+simd -mtune=cortex-a72" \
    meson -Dudev=true -Dvc4="enabled" -Dintel="disabled" -Dvmwgfx="disabled" \
    -Dradeon="disabled" -Damdgpu="disabled" -Dnouveau="disabled" -Dfreedreno="disabled" \
    -Dinstall-test-programs=true ..
$ sudo ninja install

2. Going back to Mesa

So, now let’s install Mesa. During this blogpost, I will use ${USER} as the username on the machine. Note that, in order to run sudo apt build-dep mesa, you will have to uncomment some deb-src on the file /etc/apt/sources.list and run sudo apt update.

# On the Raspberry Pi 4

# Install Mesa's build dependencies
$ sudo apt build-dep mesa

# Build and Install Mesa
$ git clone https://gitlab.freedesktop.org/mesa/mesa
$ cd mesa
$ mkdir builddir
$ mkdir installdir
$ CFLAGS="-mcpu=cortex-a72" CXXFLAGS="-mcpu=cortex-a72" \
    meson -Dprefix="/home/${USER}/mesa/installdir" -D platforms=x11 \
    -D vulkan-drivers=broadcom \
    -D gallium-drivers=kmsro,v3d,vc4 builddir
$ cd builddir
$ ninja
$ cd ..
$ ninja -C builddir install

Creating the Raspberry Pi’s sysroot

In order to cross-compile the Raspberry Pi, you need to clone the target sysroot to the host. For it, we are going to use rsync, so the host and the target need to be connected through a network.

On the Raspberry Pi 4

1. Update the system

$ sudo apt update
$ sudo apt dist-upgrade

2. Enable rsync with elevated rights

As I said before, we will be using the rsync command to sync files between the host and the Raspberry Pi. For some of these files, root rights is required internally, so let’s enable rsync with elevated rights.

$ echo "$USER ALL=NOPASSWD:$(which rsync)" | sudo tee --append /etc/sudoers

3. Setup important symlinks

Some symbolic links are needed to make the toolchain work properly, so to create all required symbolic link reliably, this bash script is needed.

$ wget https://raw.githubusercontent.com/abhiTronix/raspberry-pi-cross-compilers/master/utils/SSymlinker

Once it is downloaded, you just need to make it executable, and then run it for each path needed.

$ sudo chmod +x SSymlinker
$ ./SSymlinker -s /usr/include/aarch64-linux-gnu/asm -d /usr/include
$ ./SSymlinker -s /usr/include/aarch64-linux-gnu/gnu -d /usr/include
$ ./SSymlinker -s /usr/include/aarch64-linux-gnu/bits -d /usr/include
$ ./SSymlinker -s /usr/include/aarch64-linux-gnu/sys -d /usr/include
$ ./SSymlinker -s /usr/include/aarch64-linux-gnu/openssl -d /usr/include
$ ./SSymlinker -s /usr/lib/aarch64-linux-gnu/crtn.o -d /usr/lib/crtn.o
$ ./SSymlinker -s /usr/lib/aarch64-linux-gnu/crt1.o -d /usr/lib/crt1.o
$ ./SSymlinker -s /usr/lib/aarch64-linux-gnu/crti.o -d /usr/lib/crti.o

On the host machine

1. Setting up the directory structure

First, we need to create a workspace for building CTS, where the Raspberry Pi 4 sysroot is going to be built.

$ sudo mkdir ~/rpi-vk
$ sudo mkdir ~/rpi-vk/installdir
$ sudo mkdir ~/rpi-vk/tools
$ sudo mkdir ~/rpi-vk/sysroot
$ sudo mkdir ~/rpi-vk/sysroot/usr
$ sudo mkdir ~/rpi-vk/sysroot/usr/share
$ sudo chown -R 1000:1000 ~/rpi-vk
$ cd ~/rpi-vk

2. Sync Raspberry Pi 4 sysroot

Now, we need to sync up our sysroot folder with the system files from the Raspberry Pi. We will be using rsync that let us sync files from the Raspberry Pi. To do this, enter the following commands one by one into your terminal and remember to change username and 192.168.1.47 with the IP address of your Raspberry Pi.

$ rsync -avz --rsync-path="sudo rsync" --delete pi@192.168.1.47:/lib sysroot
$ rsync -avz --rsync-path="sudo rsync" --delete pi@192.168.1.47:/usr/include sysroot/usr
$ rsync -avz --rsync-path="sudo rsync" --delete pi@192.168.1.47:/usr/lib sysroot/usr
$ rsync -avz --rsync-path="sudo rsync" --delete pi@192.168.1.47:/usr/share sysroot/usr
$ rsync -avz --rsync-path="sudo rsync" --delete pi@192.168.1.47:/home/${USER}/mesa/installdir installdir

3. Fix symbolic links

The files we copied in the previous step still have symbolic links pointing to the file system on the Raspberry Pi. So, we need to alter this, so that they become relative links from the new sysroot directory on the host machine.

There is a Python script available online that can help us.

$ wget https://raw.githubusercontent.com/abhiTronix/rpi_rootfs/master/scripts/sysroot-relativelinks.py

Once it is downloaded, you just need to make it executable and run it.

$ sudo chmod +x sysroot-relativelinks.py
$ ./sysroot-relativelinks.py sysroot

4. Installing the Raspberry Pi 64-Bit Cross-Compiler Toolchain

As Raspbian OS 64-bits uses GCC 10.2.0, let’s install the proper cross-compiler toolchain on our host machine. I’m using the toolchain provided by abhiTronix/raspberry-pi-cross-compilers, but there are many other around the web that you can use.

We are going to use the tools folder to setup our toolchain.

$ cd ~/rpi-vk/tools
$ wget https://sourceforge.net/projects/raspberry-pi-cross-compilers/files/Bonus%20Raspberry%20Pi%20GCC%2064-Bit%20Toolchains/Raspberry%20Pi%20GCC%2064-Bit%20Cross-Compiler%20Toolchains/Bullseye/GCC%2010.2.0/cross-gcc-10.2.0-pi_64.tar.gz/download
$ tar xvf download
$ rm download

5. Setting up Wayland

If you run all the steps from this tutorial expect this one, you will still get some weird Wayland-related errors when cross-compiling it. This will happen because probably the wayland-scanner version from your host is different from the wayland-scanner version of the target. For example, on Fedora 37, the wayland-scanner version is 1.21.0 and the version on the Raspberry Pi 4 is 1.18.0.

In order to build Wayland, you will need the following dependencies:

$ sudo dnf install expat-devel xmlto

So, let’s install the proper Wayland version on our sysroot.

$ wget https://wayland.freedesktop.org/releases/wayland-1.18.0.tar.xz
$ tar xvf wayland-1.18.0.tar.xz
$ cd wayland-1.18.0
$ meson --prefix ~/rpi-vk/sysroot/usr build
$ ninja -C install

Let’s cross-compile CTS!

Now that we have the hole Raspberry Pi environment set up, we just need to create a toolchain file for CMake and its all set! So, let’s clone the CTS repository.

$ git clone https://github.com/KhronosGroup/VK-GL-CTS
$ cd VK-GL-CTS

To build dEQP, you need first to download sources for zlib, libpng, jsoncpp, glslang, vulkan-docs, spirv-headers, and spirv-tools. To download sources, run:

$ python3 external/fetch_sources.py

Inside the CTS directory, we are going to create a toolchain file called cross_compiling.cmake with the following contents:

set(CMAKE_VERBOSE_MAKEFILE ON)
set(CMAKE_SYSTEM_NAME Linux)
set(CMAKE_SYSTEM_VERSION 1)
set(CMAKE_SYSTEM_PROCESSOR aarch64)

# Check if the sysroot and toolchain paths are correct
set(tools /home/${USER}/rpi-vk/tools/cross-pi-gcc-10.2.0-64)
set(rootfs_dir $ENV{HOME}/rpi-vk/sysroot)

set(CMAKE_FIND_ROOT_PATH ${rootfs_dir})
set(CMAKE_SYSROOT ${rootfs_dir})

set(ENV{PKG_CONFIG_PATH} "")
set(ENV{PKG_CONFIG_LIBDIR} "${CMAKE_SYSROOT}/usr/lib/pkgconfig:${CMAKE_SYSROOT}/usr/share/pkgconfig")
set(ENV{PKG_CONFIG_SYSROOT_DIR} ${CMAKE_SYSROOT})

set(CMAKE_LIBRARY_ARCHITECTURE aarch64-linux-gnu)
set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} -fPIC -Wl,-rpath-link,${CMAKE_SYSROOT}/usr/lib/${CMAKE_LIBRARY_ARCHITECTURE} -L${CMAKE_SYSROOT}/usr/lib/${CMAKE_LIBRARY_ARCHITECTURE}")
set(CMAKE_C_FLAGS "${CMAKE_CXX_FLAGS} -fPIC -Wl,-rpath-link,${CMAKE_SYSROOT}/usr/lib/${CMAKE_LIBRARY_ARCHITECTURE} -L${CMAKE_SYSROOT}/usr/lib/${CMAKE_LIBRARY_ARCHITECTURE}")
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fPIC -Wl,-rpath-link,${CMAKE_SYSROOT}/usr/lib/${CMAKE_LIBRARY_ARCHITECTURE} -L${CMAKE_SYSROOT}/usr/lib/${CMAKE_LIBRARY_ARCHITECTURE}")

set(WAYLAND_SCANNER ${CMAKE_SYSROOT}/usr/bin/wayland-scanner)

## Compiler Binary
SET(BIN_PREFIX ${tools}/bin/aarch64-linux-gnu)

SET (CMAKE_C_COMPILER ${BIN_PREFIX}-gcc)
SET (CMAKE_CXX_COMPILER ${BIN_PREFIX}-g++ )
SET (CMAKE_LINKER ${BIN_PREFIX}-ld
            CACHE STRING "Set the cross-compiler tool LD" FORCE)
SET (CMAKE_AR ${BIN_PREFIX}-ar
            CACHE STRING "Set the cross-compiler tool AR" FORCE)
SET (CMAKE_NM {BIN_PREFIX}-nm
            CACHE STRING "Set the cross-compiler tool NM" FORCE)
SET (CMAKE_OBJCOPY ${BIN_PREFIX}-objcopy
            CACHE STRING "Set the cross-compiler tool OBJCOPY" FORCE)
SET (CMAKE_OBJDUMP ${BIN_PREFIX}-objdump
            CACHE STRING "Set the cross-compiler tool OBJDUMP" FORCE)
SET (CMAKE_RANLIB ${BIN_PREFIX}-ranlib
            CACHE STRING "Set the cross-compiler tool RANLIB" FORCE)
SET (CMAKE_STRIP {BIN_PREFIX}-strip
            CACHE STRING "Set the cross-compiler tool RANLIB" FORCE)

set(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER)
set(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY)
set(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY)

Note that we had to specify our toolchain and also the specify the path to the wayland-scanner. Now that we are all set, we can finally cross-compile CTS.

$ mkdir build
$ cd build
$ cmake -DCMAKE_BUILD_TYPE=Debug \
	-DCMAKE_LIBRARY_PATH=/home/${USER}/rpi-vk/installdir/lib \
	-DCMAKE_INCLUDE_PATH=/home/${USER}/rpi-vk/installdir/include \
	-DCMAKE_GENERATOR=Ninja \
	-DCMAKE_TOOLCHAIN_FILE=/home/${USER}/VK-GL-CTS/cross_compiling.cmake ..
$ ninja

Now, you can transfer the compiled files to the Raspberry Pi 4 and run CTS!

This was a fun little challenge of my CE project and it was pretty nice to learn more about CTS. Running CTS was also a great idea from Melissa as I was able to hexdump the contents of a noop job for the V3DV and fix my noop job on IGT. So, now I finally have a working noop job on IGT and you can check it here.

Also, a huge thanks to my friend Arthur Grillo for helping me with resources about cross-compiling for the Raspberry Pi.

Rotating Planes on VKMS

Mon, 08 May 2023 00:00:00 +0000

In my last blog post, I described a bit of my previous work on the rustgem project, and after that, as I had finished the VGEM features, I sent a RFC to the mailing list. Although I still need to work on some rustgem feedback, I started to explore more of the KMS (Kernel Mode Setting) and its properties.

I talked to my mentor Melissa Wen, one of the VKMS maintainers, and she proposed implementing plane rotation capabilities to VKMS. The VKMS (Virtual Kernel Mode Setting) is a software-only KMS driver that is quite useful for testing and running X (or similar compositors) on headless machines. It sounded like a great idea, as I would like to explore a bit more of the KMS side of things.

What is Plane Rotation?

In order to have an image on a display, we need to go through the whole Kernel Mode Setting (KMS) Display Pipeline. The pipeline has a couple of different objects, such as framebuffers, planes, and CRTCs, and the relationship between them can be quite complicated. If you are interested in the KMS Display Pipeline, I recommend reading the great KMS documentation. But here we are focused in only one of those abstractions, the plane.

In the context of graphics processing, a plane refers to an image source that can be superimposed or blended on top of a CRTC during the scanout process. The plane itself specifies the cropping and scaling of that image, and where it is placed on the visible area of the CRTC. Moreover, planes may possess additional attributes that dictate pixel positioning and blending, such as rotation or Z-positioning.

Rotation is an optional KMS property of the DRM plane object, which we use to specify the rotation amount in degrees in counter-clockwise direction. The rotation is applied to the image sampled from the source rectangle, before scaling it to fit in the destination rectangle. So, basically, the rotation property adds a rotation and a reflection step between the source and destination rectangles.

		|*********|$$$$$$$$$|              |$$$$$$$$$|@@@@@@@@@|
		|*********|$$$$$$$$$|  --------->  |$$$$$$$$$|@@@@@@@@@|
		|#########|@@@@@@@@@|     90º      |*********|#########|
		|#########|@@@@@@@@@|              |*********|#########|

The possible rotation values are rotate-0, rotate-90, rotate-180, rotate-270, reflect-x and reflect-y.

Now that we understand what plane rotation is, we can think about how to implement the rotation property on VKMS.

Rotation on VKMS

VKMS has some really special driver attributes, as all its composition happens by software operations. The rotation is usually an operation that is performed on the user-space, but the hardware can also perform it. In order for the hardware to perform it, the driver will set some registers, change some configurations, and indicate to the hardware that the plane should be rotated. This doesn’t happen on VKMS, as the composition is essentially a software loop. So, we need to modify this loop to perform the rotation.

First, we need a brief notion of how the composition happens in VKMS. The composition in VKMS happens line-by-line. Each line is represented by a staging buffer, which contains the composition for one plane, and an output buffer, which contains the composition of all planes in z-pos order. For each line, we query an array by the first pixel of the line and go through the whole source array linearly, performing the proper pixel conversion. The composition of the line can be summarized by:

void vkms_compose_row(struct line_buffer *stage_buffer, struct vkms_plane_state *plane, int y)
{
	struct pixel_argb_u16 *out_pixels = stage_buffer->pixels;
	struct vkms_frame_info *frame_info = plane->frame_info;
	u8 *src_pixels = get_packed_src_addr(frame_info, y);
	int limit = min_t(size_t, drm_rect_width(&frame_info->dst), stage_buffer->n_pixels);

	for (size_t x = 0; x < limit; x++, src_pixels += frame_info->cpp)
		plane->pixel_read(src_pixels, &out_pixels[x]);
}

Here we can see that we have the line, represented by the stage buffer and the y coordinate, and the source pixels. We read each source pixel in a linear manner, through the for-loop, and we place it on the stage buffer in the appropriate format.

With that in mind, we can think that rotating a plane is a matter of changing how we read and interpret the lines. Let’s think about the reflect-x operation.

		|*********|$$$$$$$$$|                |$$$$$$$$$|*********|
		|*********|$$$$$$$$$|  ----------->  |$$$$$$$$$|*********|
		|#########|@@@@@@@@@|   reflect-x    |@@@@@@@@@|#########|
		|#########|@@@@@@@@@|                |@@@@@@@@@|#########|

Thinking that the VKMS composition happens line-by-line, we can describe the operation as a read in reverse order. So, instead of start reading the pixels from left to right, we need to start reading the pixels from right to left. We can implement this by getting the limit of the line and subtracting the current x position:

static int get_x_position(const struct vkms_frame_info *frame_info, int limit, int x)
{
	if (frame_info->rotation & DRM_MODE_REFLECT_X)
		return limit - x - 1;
	return x;
}

For the reflect-y operation, we need to start reading the plane from the last line, instead of reading it from the first line.

		|*********|$$$$$$$$$|                |#########|@@@@@@@@@|
		|*********|$$$$$$$$$|  ----------->  |#########|@@@@@@@@@|
		|#########|@@@@@@@@@|   reflect-y    |*********|$$$$$$$$$|
		|#########|@@@@@@@@@|                |*********|$$$$$$$$$|

This can be performed by changing the y on the external composition loop. Similarly from the reflect-x case, we can get the y limit and subtract the current y position.

static int get_y_pos(struct vkms_frame_info *frame_info, int y)
{
	if (frame_info->rotation & DRM_MODE_REFLECT_Y)
		return drm_rect_height(&frame_info->rotated) - y - 1;
	return y;
}

So, to implement the rotation in VKMS, we need to change how we interpret the boundaries of the plane and read accordingly.

This might seem odd because we could just rotate the src rectangle by using drm_rect_rotate, but this wouldn’t work as the composition in VKMS is performed line-by-line and the pixels are accessed linearly. However, drm_rect_rotate is of great help for us on the rotate-90 and rotate-270 cases. Those cases demand scaling and drm_rect_rotate helps us tremendously with it. Basically, what it does is:

		                                              |$$|@@|
		                                              |$$|@@|
		|*********|$$$$$$$$$|                         |$$|@@|
		|*********|$$$$$$$$$|  -------------------->  |$$|@@|
		|#########|@@@@@@@@@|   drm_rect_rotate(90)   |**|##|
		|#########|@@@@@@@@@|                         |**|##|
		                                              |**|##|
		                                              |**|##|

After the drm_rect_rotate operation, we need to read the columns as lines and the lines as columns. See that even for a case like rotate-90, it is just a matter of changing the point of view and reading the lines differently.

The complete implementation of all rotation modes is available here. Together with the rotation feature, I sent a patch to reduce the code repetition in the code by isolating the pixel conversion functionality. This patch was already merged, but the rest of the series is still pending a Reviewed-by.

Rotating planes on VKMS was a fun challenge of my Igalia Coding Experience and I hope to keep working on VKMS to bring more and more features.

Adding a Timeout feature to Rustgem

Wed, 22 Mar 2023 00:00:00 +0000

After my last blogpost, I kept developing the Rust version of the VGEM driver, also known as rustgem for now. Previously, I had developed two important features of the driver: the ability to attach a fence and the ability to signal a fence. Still one important feature is still missing: the ability to prevent hangs. Currently, if the fence is not signaled, the driver will simply hang. So, we can create a callback that signals the fence when the fence is not signaled by the user for more than 10 seconds.

In order to create this callback, we need to have a Timer that will trigger it after the specified amount of time. Gladly, the Linux kernel provides us with a Timer that can be set with a callback and a timeout. But, to use it in the Rust code, we need to have a safe abstraction, that will ensure that the code is safe under some assumptions.

First Attempt: writing a Timer abstraction

Initially, I was developing an abstraction on my own as I checked the RfL tree and there were no Timer abstractions available.

The most important question here is “how can we guarantee access to other objects inside the callback?”. The callback only has receives a pointer to the struct timer_list as its single argument. Naturally, we can think about using a container_of macro. In order to make the compatibility layer between Rust and the C callback, I decided to store the object inside the Timer. Yep, I didn’t like that a lot, but it was the solution I came up with at the time. The struct looked something like this:

/// A driver-specific Timer Object
//
// # Invariants
// timer is a valid pointer to a struct timer_list and we own a reference to it.
[repr(C)]
pub struct UniqueTimer<T: TimerOps<Inner = D>, D> {
   timer: *bindings::timer_list,
   inner: D,
   _p: PhantomData<T>,
}

Moreover, the second important question I had was “how can the user pass a callback function to the timer?”. There were two possibilities: using a closure and using a Trait. I decided to go through the trait path. Things would be kind of similar if I decided to go into the closure path.

/// Trait which must be implemented by driver-specific timer objects.
pub trait TimerOps: Sized {
    /// Type of the Inner data inside the Timer
    type Inner;

    /// Timer callback
    fn timer_callback(timer: &UniqueTimer<Self, Self::Inner>);
}

With those two questions solved, it seems that we are all set and good to go. So, we can create methods to initialize the timer and modify the timer’s timeout, implement the Drop trait, and use the following callback by default:

unsafe extern "C" fn timer_callback<T: TimerOps<Inner = D>, D: Sized>(
    timer: *mut bindings::timer_list,
) {
    let timer = crate::container_of!(timer, UniqueTimer<T, D>, timer)
			    as *mut UniqueTimer<T, D>;

    // SAFETY: The caller is responsible for passing a valid timer_list subtype
    T::timer_callback(unsafe { &mut *timer });
}

All should work, right? Well… No, I didn’t really mention how I was allocating memory. And let’s say I was initially allocating it wrongly and therefore, the container_of macro was pointing to the wrong memory space.

Initially, I was allocating only timer with the kernel memory allocator krealloc and allocating the rest of the struct with Rust’s memory allocator. By making such a mess, container_of wasn’t able to point to the right memory address.

I had to change things a bit to allocate the whole struct UniqueTimer with the kernel’s memory allocator. However, krealloc returns a raw pointer and it would be nice for the final user to get a raw pointer to the object. I wrapped up inside another struct that could be dereferenced into the UniqueTimer object.

/// A generic Timer Object
///
/// This object should be instantiated by the end user, as it holds
/// a unique reference to the UniqueTimer struct. The UniqueTimer
/// methods can be used through it.
pub struct Timer<T: TimerOps<Inner = D>, D>(*mut UniqueTimer<T, D>);

impl<T: TimerOps<Inner = D>, D> Timer<T, D> {
    /// Create a timer for its first use
    pub fn setup(inner: D) -> Self {
        let t = unsafe {
            bindings::krealloc(
                core::ptr::null_mut(),
                core::mem::size_of::<UniqueTimer<T, D>>(),
                bindings::GFP_KERNEL | bindings::__GFP_ZERO,
            ) as *mut UniqueTimer<T, D>
        };

        // SAFETY: The pointer is valid, so pointers to members are too.
        // After this, all fields are initialized.
        unsafe {
            addr_of_mut!((*t).inner).write(inner);
            bindings::timer_setup(addr_of_mut!((*t).timer), Some(timer_callback::<T, D>), 0)
        };

        Self(t)
    }
}

And then the container_of macro started working! Now, I could setup a Timer for each fence and keep the fence inside the timer. Finally, I could use the fence inside the timer to signal it when it was not signaled by the user for more than 10 seconds.

impl TimerOps for VgemFenceOps {
    type Inner = UniqueFence<Self>;

    fn timer_callback(timer: &UniqueTimer<Self, UniqueFence<Self>>) {
        let _ = timer.inner().signal();
    }
}

So, I tested the driver with IGT using the vgem_slow test and it was now passing! All IGT tests were passing and it looked like the driver was practically completed (some FIXME problems notwithstanding). But, let’s see if this abstraction is really safe…

Second Attempt: using a Timer abstraction

First, let’s inspect the struct timer_list in the C code.

struct timer_list {
    struct hlist_node   entry;
    unsigned long       expires;
    void            (*function)(struct timer_list *);
    u32         flags;
};

By looking at this struct, we can see a problem in my abstraction: a timer can point to a timer through a list. If you are not familiar with Rust, this can seem normal, but self-referential types can lead to undefined behavior (UB).

Let’s say we have an example type with two fields: u32 and a pointer to this u32 value. Initially, everything looks fine, the pointer field points to the value field in memory address A, which contains a valid u32, and all pointers are valid. But Rust has the freedom to move values around memory. For example, if we pass this struct into another function, it might get moved to a different memory address. So, the once valid pointer is no longer valid, because when we move the struct, the struct’s fields change their address, but not their value. Now, the pointer fields still point to the memory address A, although the value field is located at the memory address B now. This is really bad and can lead to UB.

The solution is to make timer_list implement the !Unpin trait. This means that to use this type safely, we can’t use regular pointers for self-reference. Instead, we use special pointers that “pin” their values into place, ensuring they can’t be moved.

Still looking at the struct timer_list, it is possible to notice that a timer can queue itself in the timer function. This functionality is not covered by my current abstraction.

Moreover, I was using jiffies to modify the timeout duration and I was adding a Duration to the jiffies. This is problematic, because it can cause a data races. Reading jiffies and adding a duration to them should be an atomic operation.

Huge thanks to the RfL folks that pointed the errors in my implementation!

With all these problems pointed out, it is time to fix them! I could have reimplemented my safe abstraction, but the RfL folks pointed me to a Timer abstraction that they are developing in a downstream tree. Therefore, I decided to use their Timer abstraction.

There were two options to implement a Timer abstraction:

To implement the Timeout trait to the VgemFence struct
To use the FnTimer abstraction

In the end, I decided to go with the second approach. The FnTimer receives a closure that will be executed at the timeout. The closure can return an enum that indicated if the timer is done or if it should be rescheduled.

When implementing the timer, I had a lot of borrow checker problems. See… I need to use the Fence object inside the callback and also move the Fence object at the end of the function. So, I got plenty of “cannot move out of fence because it is borrowed” errors. Also, I needed the Timer to be dropped at the same time as the fence, so I needed to store the Timer inside the VgemFence struct.

The solution to the problems: smart pointers! I boxed the FnTimer and the closure inside the FnTimer so that I could store it inside the VgemFence struct. Then, the second problem got fixed. But, I still cannot use the fence inside the closure, because it wasn’t encapsulated inside a smart pointer. So, I used an Arc to box Fence, clone it, and move it to the scope of the closure.

pub(crate) struct VgemFence {
	fence: Arc<UniqueFence<Fence>>,
	_timer: Box<FnTimer<Box<dyn FnMut() -> Result<Next> + Sync>>>,
}

impl VgemFence {
	pub(crate) fn create() -> Result<Self> {
		let fence_ctx = FenceContexts::new(1, QUEUE_NAME, &QUEUE_CLASS_KEY)?;
		let fence = Arc::try_new(fence_ctx.new_fence(0, Fence {})?)?;

		// SAFETY: The caller calls [`FnTimer::init_timer`] before using the timer.
		let t = Box::try_new(unsafe {
			FnTimer::new(Box::try_new({
				let fence = fence.clone();
				move || {
					let _ = fence.signal();
					Ok(Next::Done)
				}
			})? as Box<_>)
		})?;

		// SAFETY: As FnTimer is inside a Box, it won't be moved.
		let ptr = unsafe { core::pin::Pin::new_unchecked(&*t) };

		timer_init!(ptr, 0, "vgem_timer");

		// SAFETY: Duration.as_millis() returns a valid total number of whole milliseconds.
		let timeout =
			unsafe { bindings::msecs_to_jiffies(Duration::from_secs(10).as_millis().try_into()?) };

		// We force the fence to expire within 10s to prevent driver hangs
		ptr.raw_timer().schedule_at(jiffies_later(timeout));

		Ok(Self { fence, _timer: t })
	}
}

You can observe in this code that the initialization of the FnTimer uses an unsafe operation. This happens because we still don’t have Safe Pinned Initialization. But the RfL folks are working hard to land this feature and improve ergonomics when using Pin.

Now, running again the vgem_slow IGT test, you can see that all IGT tests are now passing!

Next Steps

During this time, many improvements landed in the driver: all the objects are being properly dropped, including the DRM device; all error cases are returning the correct error; the SAFETY comments are properly written and most importantly, the timeout feature was introduced. With that, all IGT tests are passing and the driver is functional!

Now, the driver is in a good shape, apart from one FIXME problem: currently, the IOCTL abstraction doesn’t support any drivers that the IOCTLs don’t start in 0x00 and the VGEM driver starts its IOCTLs with 0x01. I don’t know yet how to bypass this problem without adding a dummy IOCTL as 0x00, but I hope to get a solution to it soon.

The progress of this project can be followed in this PR and I hope to see this project being integrated upstream in the future.

Rust for VGEM

Tue, 28 Feb 2023 00:00:00 +0000

In the last blog post, I pointed out that I didn’t know exactly what it would be my next steps for the near future. Gladly, I had the amazing opportunity to start a new Igalia Coding Experience with a new project.

This time Melissa Wen pitched me with the idea to play around with Rust for Linux in order to rewrite the VGEM driver in Rust. The Rust for Linux project is growing fast with new bindings and abstractions being introduced in the downstream RfL kernel. Also, some basic functionalities were introduced in Linux 6.1. Therefore, it seems like a great timing to start exploring Rust in the DRM subsystem!

Why Rust?

As mentioned by the Rust website, using Rust means Performance, Reliability, and Productivity. Rust is a blazingly fast and memory-efficient language with its powerful ownership model. No more looking for use-after-free and memory leaks, as Rust guarantees memory safety and thread safety, eliminating a handful of bugs at compile-time.

Moreover, Rust provides a new way of programming. The language provides beautiful features such as traits, enums, and error handling, that can make us feel empowered by the language. We can use a lot of concepts from functional programming and mix them with concepts from OOP, for example.

Although I’m an absolute beginner in Rust, I can see the major advantages of the Rust programming language. From the start, it was a bit tough to enjoy the language, as I was fighting with the compiler most of the time. But now that I have a more firm foundation on Rust, it is possible to appreciate the beauty in Rust and I don’t see myself starting a new project in C++ for a long while.

Bringing Rust to the Linux Kernel is a ambitious idea, but it can lead to great changes. We can think about a world where no developers are looking for memory leaks and use-after-free bugs due to the safety that Rust can provide us.

Rust on DRM

Now, what about Rust for DRM? I mean, I’m not the first one to think about it. Asahi Lina is making a fantastic work on the Apple M1 GPU and things are moving quite fast there. She already had great safe abstractions for the DRM bindings and provides us the very basis for anyone who is willing to start a new DRM driver in Rust, which is my case.

That said, why not make use of Lina’s excellent bindings to build a new driver?

Rust for VGEM

VGEM (Virtual GEM Provider) is a minimal non-hardware backed GEM (Graphics Execution Manager) service. It is used with non-native 3D hardware for buffer sharing between the X server and DRI. It is a fairly simple driver with about 400 lines of code and it uses the DMA Fence API to handle attaching and signaling the fences.

So, to rewrite VGEM in Rust, some bindings are needed, e.g. bindings for platform device, for XArray, and for dealing with DMA fence and DMA reservations. Furthermore, many DRM abstractions are needed as well.

In this sense, a lot of the DRM abstractions are already developed by Lina and also she is developing abstractions for DMA fence. So, in this project, I’ll be focusing on the bindings that Lina and the RfL folks haven’t developed yet.

After developing the bindings, it is a matter of developing the driver, which it’ll be quite simple after all DMA abstractions are set, because most of the driver consists of fence manipulation.

Current Status

I have developed the main platform device registration of the driver. As VGEM is a virtual device, the standard probe initialization is not useful, as a virtual device cannot be probed by the pseudo-bus that holds the platform devices. So, as VGEM is not a usual hotplugged device, we need to use the legacy platform device initialization. This made me develop my first binding for legacy registration:

/// Add a platform-level device and its resources
pub fn register(name: &'static CStr, id: i32) -> Result<Self> {
	let pdev = from_kernel_err_ptr(unsafe {
		bindings::platform_device_register_simple(name.as_char_ptr(), id,
			core::ptr::null(), 0)
	})?;

	Ok(Self {
		ptr: pdev,
		used_resource: 0,
		is_registered: true,
	})
}

For sure, the registration must follow the unregistration of the device, so I implemented a Drop trait for the struct Device in order to guarantee the proper device removal without explicitly calling it.

impl Drop for Device {
	fn drop(&mut self) {
		if self.is_registered {
			// SAFETY: This path only runs if a previous call to `register`
			// completed successfully.
			unsafe { bindings::platform_device_unregister(self.ptr) };
		}
	}
}

After those, I also developed bindings for a couple of more functions and together with Lina’s bindings, I could initialize the platform device and register the DRM device under a DRM minor!

[   38.825684] vgem: vgem_init: platform_device with id -1
[   38.826505] [drm] Initialized vgem 1.0.0 20230201 for vgem on minor 0
[   38.858230] vgem: Opening...
[   38.862377] vgem: Closing...
[   41.543416] vgem: vgem_exit: drop

Next, I focused on the development of the two IOCTLs: drm_vgem_fence_attach and drm_vgem_fence_signal. The first is responsable for creating and attaching a fence to the VGEM handle, while the second signals and consumes a fence earlier attached to a VGEM handle.

In order to add a fence, bindings to DMA reservation are needed. So, I started by creating a safe abstraction for struct dma_resv.

/// A generic DMA Resv Object
///
/// # Invariants
/// ptr is a valid pointer to a dma_resv and we own a reference to it.
pub struct DmaResv {
    ptr: *mut bindings::dma_resv,
}

impl DmaResv {
	
    [...]
	
    /// Add a fence to the dma_resv object
    pub fn add_fences(
        &self,
        fence: &dyn RawDmaFence,
        num_fences: u32,
        usage: bindings::dma_resv_usage,
    ) -> Result {
        unsafe { bindings::dma_resv_lock(self.ptr, core::ptr::null_mut()) };

        let ret = self.reserve_fences(num_fences);
        match ret {
            Ok(_) => {
                // SAFETY: ptr is locked with dma_resv_lock(), and dma_resv_reserve_fences()
                // has been called.
                unsafe {
                    bindings::dma_resv_add_fence(self.ptr, fence.raw(), usage);
                }
            }
            Err(_) => {}
        }
        
        unsafe { bindings::dma_resv_unlock(self.ptr) };

        ret
    }
}

With that step, I could simply write the IOCTLs based on the new DmaResv abstraction and Lina’s fence abstractions.

To test the IOCTLs, I used some already available IGT tests: dmabuf_sync_file and vgem_basic. Those tests use VGEM as it base, so if the tests pass, it means that the IOCTLs are working properly. And, after some debugging and rework in the IOCTLs, I managed to get most of the tests to pass!

[root@fedora igt-gpu-tools]# ./build/tests/dmabuf_sync_file
IGT-Version: 1.27-gaa16e812 (x86_64) (Linux: 6.2.0-rc3-asahi-02441-g6c8eda039cfb-dirty x86_64)
Starting subtest: export-basic
Subtest export-basic: SUCCESS (0.000s)
Starting subtest: export-before-signal
Subtest export-before-signal: SUCCESS (0.000s)
Starting subtest: export-multiwait
Subtest export-multiwait: SUCCESS (0.000s)
Starting subtest: export-wait-after-attach
Subtest export-wait-after-attach: SUCCESS (0.000s)

You can check out the current progress of this project on this pull request.

Next Steps

Although most of the IGT tests are now passing, two tests aren’t working yet: vgem_slow, as I haven’t introduced the timeout yet, and vgem_basic@unload, as I still need to debug why the Drop trait from drm::drv::Registration is not being called.

After bypassing those two problems, I still need to rework some of my code, as, for example, I’m using a dummy IOCTL as IOCTL number 0x00, as the current macro kernel::declare_drm_ioctl doesn’t support any drivers for which the IOCTL doesn’t start in 0x00.

So, there is a lot of work yet to be done!

January Update: Finishing my Igalia CE

Tue, 17 Jan 2023 00:00:00 +0000

2022 really passed by fast and after I completed the GSoC 2022, I’m now completing another milestone: my project in the Igalia Coding Experience and I had the best experience during those four months. I learned tremendously about the Linux graphics stack and now I can say for sure that I would love to keep working in the DRM community.

While GSoC was, for me, an experience to get a better understanding of what open source is, Igalia CE was an opportunity for me to mature my knowledge of technical concepts.

So, this is a summary report of my journey at the Igalia CE.

IGT tests to V3D

Initially, V3D only had three basic IGT tests: v3d_get_bo_offset, v3d_get_param, and v3d_mmap. So, the basic goal of my CE project was to add more tests to the V3D driver.

V3D is the driver that supports the Broadcom V3D 3.3 and 4.1 OpenGL ES GPUs, and is the driver that provides 3D rendering to the Raspberry Pi 4. V3D is composed of a tiled renderer, a TFU (Texture Formatting Unit), and a CSD (Compute Shader Dispatch).

During the CE, I was able to develop tests for almost all eleven V3D ioctls (except v3d_submit_tfu). I began writing tests to the v3d_create_bo ioctl and Performance Monitor (perfmon) related ioctls. I developed tests that check the basic functionality of the ioctls and I inspected the kernel code to understand situations where the ioctl should fail.

After those tests, I got the biggest challenge that I had on my CE project: performing a Mesa’s no-op job on IGT. A no-op job is one of the simplest jobs that can be submitted to the V3D. It is a 3D rendering job, so it is a job submitted through the v3d_submit_cl ioctl, and performing this job on IGT was fundamental to developing good tests for the v3d_submit_cl ioctl.

The main problem I faced on submitting a no-op job on IGT was: I would copy many and many Mesa files to IGT. And I took a while fighting against this idea, looking for other ways to submit a job to V3D. But, as some experience developers pointed out, packeting is the best option for it. So indeed, the final solution I came in with was to copy a couple of files from Mesa, but just three of them, which sounds reasonable.

So, after some time, I was able to bring the Mesa structure to IGT with minimal overhead. But, I was still not able to run a successful no-op job as the job’s fence wasn’t being signaled by the end of the job. Then, Melissa Wen guided me to experiment running CTS tests to inspect the no-op job. With the CTS tests, I was able to hexdump the contents of the packet and understand what was going on wrong in my no-op job.

Running the CTS in the Raspberry Pi 4 was a fun side-quest of the project and ended up resulting in a commit to the CTS repository, as CTS wasn’t handling appropriately the wayland-scanner for cross-compiling. CTS was picking the wayland-scanner from the host computer instead of picking the wayland-scanner executable available in the target sysroot. This was fixed with this simple patch:

Allow override of wayland_scanner executable

When I finally got a successful no-op job, I was able to write the tests for the v3d_submit_cl and v3d_wait_bo ioctls. On these tests, I tested primarily job synchronization with single syncobjs and multiple syncobjs. In this part of the project, I had the opportunity to learn a lot about syncobjs and different forms of synchronization in the kernel and userspace.

Having done the v3d_submit_cl tests, I developed the v3d_submit_csd tests in a similar way, as the job submission process is kind of similar. For submitting a CSD job, it is necessary to make a valid submission with a pipeline assembly shader and as IGT doesn’t have a shader compiler, so I hard-coded the assembly of an empty shader in the code. In this way, I was able to get a simple CSD job submitted, and having done that, I could now play around with mixing CSD and CL jobs.

In these tests, I could test the synchronization between two job queues and see, for example, if they were proceeding independently.

So, by the end of the review process, I will add 66 new sub-tests to V3D, having in total 72 IGT sub-tests! Those tests are checking invalid parameters, synchronization, and the proper behavior of the functionalities.

Patch/Series	Status
[PATCH 0/7] V3D IGT Tests Updates	Accepted
[PATCH 0/2] Tests for V3D/VC4 Mmap BO IOCTLs	Accepted
[PATCH 0/4] Make sure v3d/vc4 support performance monitor	In Review
[PATCH 0/6] V3D Job Submission Tests	In Review
[PATCH 0/3] V3D Mixed Job Submission Tests	In Review

Mesa

Apart from reading a lot of kernel code, I also started to explore some of the Mesa code, especially the v3dv driver. On Mesa, I was trying to understand the userspace use of the ioctls in order to create useful tests. While I was exploring the v3dv, I was able to make two very simple contributions to Mesa: fixing typos and initializing a variable in order to assure proper error handling.

Patch	Status
v3dv: fix multiple typos	Accepted
v3dv: initialize fd variable for proper error handling	Accepted

IGT tests to VC4

VC4 and V3D share some similarities in their basic 3D rendering implementation. VC4 contains a 3D engine, and a display output pipeline that supports different outputs. The display part of the VC4 is used on the Raspberry Pi 4 together with the V3D driver.

Although my main focus was on the V3D tests, as the VC4 and V3D drivers are kind of similar, I was able to bring some improvements to the VC4 tests as well. I added tests for perfmons and the vc4_mmap ioctl and improved a couple of things in the tests, such as moving it a separate folder and creating a check to skip the VC4 tests if they are running on a Raspberry Pi 4.

Patch/Series	Status
[PATCH 0/5] VC4 IGT Tests Updates	Accepted
[PATCH 0/2] Tests for V3D/VC4 Mmap BO IOCTLs	Accepted
[PATCH 0/4] Make sure v3d/vc4 support performance monitor	In Review
tests/vc4_purgeable_bo: Fix conditional assertion	In Review

Linux Kernel

V3D/VC4 drivers

During this process of writing tests to IGT, I ended up reading a lot of kernel code from V3D in order to evaluate possible userspace scenarios. While inspecting some of the V3D code, I could find a couple of small things that could be improved, such as using the DRM-managed API for mutexes and replacing open-coded implementations with their DRM counterparts.

Patch	Status
drm/v3d: switch to drmm_mutex_init	Accepted
drm/v3d: add missing mutex_destroy	Accepted
drm/v3d: replace open-coded implementation of drm_gem_object_lookup	Accepted

Although I didn’t explore the VC4 driver as much as the V3D driver, I also took a look at the driver, and I was able to detect a small thing that could be improved: using the DRM-core helpers instead of open-code. Moreover, after a report on the mailing list, I bisected a deadlock and I was able to fix it after some study about the KMS locking system.

Patch	Status
drm/vc4: drop all currently held locks if deadlock happens	Accepted
drm/vc4: replace drm_gem_dma_object for drm_gem_object in vc4_exec_info	In Review
drm/vc4: replace obj lookup steps with drm_gem_objects_lookup	In Review

The debugfs side-quest

The debugfs side-quest was a total coincidence during this project. I had some spare time and was looking for something to develop. While looking at the DRM TODO list, I bumped into the debugfs clean-up task and found it interesting to work on. So, I started to work on this task based on the previous work from Wambui Karuga, who was a Outreachy mentee and worked on this feature during her internship. By chance, when I talked to Melissa about it, she told me that she had knowledge of this project due to a past Outreachy internship that she was engaged on, and she was able to help me figure out the last pieces of this side-quest.

After submitting the first patch, introducing the debugfs device-centered functions, and converting a couple of drivers to the new structure, I decided to remove the debugfs_init hook from a couple of drivers in order to get closer to the goal of removing the debugfs_init hook. Moreover, during my last week in the CE, I tried to write a debugfs infrastructure for the KMS objects, which was another task in the TODO list, although I still need to do some rework on this series.

Patch/Series	Status
[PATCH 0/7] Introduce debugfs device-centered functions	Accepted
drm/debugfs: use octal permissions instead of symbolic permissions	Accepted
drm/debugfs: add descriptions to struct parameters	Accepted
[PATCH 0/7] Convert drivers to the new debugfs device-centered functions	Accepted
[PATCH 0/13] drm/debugfs: Create a debugfs infrastructure for kms objects	In Review

More side-quests

By the end of the CE, I was on my summer break from university, so I had some time to take a couple of side-quests in this journey.

The first side-quest that I got into originated in a failed IGT test on the VC4, the “addfb25-bad-modifier” IGT test. Initially, I proposed a fix only for the VC4, but after some discussion in the mailing list, I decided to move forward with the idea to create the check for valid modifiers in the DRM core. The series is still in review, but I had some great interactions during the iterations.

The second side-quest was to understand why the IGT test kms_writeback was causing a kernel oops in vkms. After some bisecting and some study about KMS’s atomic API, I was able to detect the problem and write a solution for it. It was pretty exciting to deal with vkms for the first time and to get some notion about the display side of things.

Patch/Series	Status
drm/tests: Split drm_test_dp_mst_calc_pbn_mode into parameterized tests	Accepted
drm/tests: Split drm_test_dp_mst_sideband_msg_req_decode into parameterized tests	Accepted
tests/kms_addfb_basic: Avoid open-coded expressions	Accepted
[PATCH 0/3] Check for valid framebuffer’s format	In Review
drm/vkms: reintroduce prepare_fb and cleanup_fb functions	Accepted

Next Steps

A bit different from the end of GSoC, I’m not really sure what are going to be my next steps in the next couple of months. The only thing I know for sure is that I will keep contributing to the DRM subsystem and studying more about DRI, especially the 3D rendering and KMS.

The DRI infrastructure is really fascinating and there is so much to be learn! Although I feel that I improved a lot in the last couple of months, I still feel like a newbie in the community. I still want to have more knowledge of the DRM core helpers and understand how everything glues together.

Apart from the DRM subsystem, I’m also trying to take some time to program more in Rust and maybe contribute to other open-source projects, like Mesa.

Acknowledgment

I would like to thank my great mentors Melissa Wen and André Almeida for helping me through this journey. I wouldn’t be able to develop this project without their great support and encouragement. They were an amazing duo of mentors and I thank them for answering all my questions and helping me with all the challenges.

Also, I would like to thank the DRI community for reviewing my patches and giving me constructive feedback. Especially, I would like to thank Daniel Vetter for answering patiently every single question that I had about the debugfs clean-up and to thank Jani Nikula, Maxime Ripard, Thomas Zimmermann, Javier Martinez Canillas, Emma Anholt, Simon Ser, Iago Toral, Kamil Konieczny and many others that took their time to review my patches, answer my questions and provide me constructive feedback.

November Update: Exploring V3D

Thu, 08 Dec 2022 00:00:00 +0000

It has been a busy couple of months. As I pointed on my last blog post, I finished GSoC and joined the Igalia Coding Experience mentorship project. In October, I also traveled to Minneapolis for XDC 2022 where I presented to the Linux Graphics community our AMD/KUnit work with my colleagues. So, let’s make a summary of the last couple of months.

XDC 2022

Just a small thank you note to X.Org Foundation for sponsoring my travel to Minneapolis. XDC 2022 was a great experience, and I learned quite a lot during the talks. Although I was a newcomer, all developers were very nice to me, and it was great to talk to experienced developers (and meet my mentors in person). Also, I presented the GSoC/XOrg work on the first day of the conference and this talk is available on YouTube.

Working with the Raspberry Pi 4

As I mentioned in my last blog post, GSoC was a great learning experience and I’m willing to keep learning about the Linux graphics stack. Fortunately, when I started the Igalia CE, Melissa Wen pitched me a project to increase IGT test coverage on DRM/V3D kernel driver. I was pretty glad to hear about the project as it allowed me to learn more about how a GPU works.

The Project

Currently, V3D only has three basic IGT tests: v3d_get_bo_offset, v3d_get_param, and v3d_mmap. So, the basic goal of my CE project was to add more tests to the V3D driver.

As the general DRM-core tests were in a good shape on the V3D driver, I started to think together with my mentors about more driver-specific tests for the driver.

By checking the V3D UAPI, you can see that the V3D has eleven ioctls, so there is yet a lot to test for the V3D on IGT.

First, there are Buffer Object (BO) related-ioctls: v3d_create_bo, v3d_wait_bo, v3d_mmap_bo, and v3d_get_bo_offset. The Buffer Objects are shared-memory objects that are allocated by the GPU to store things like vertex data. Therefore, testing them is important to make sure that memory is being correctly allocated. Different from the VC4, the V3D has an MMU between the GPU and the bus, allowing it to not allocate objects contiguously. Therefore, the idea was to develop tests for v3d_create_bo and v3d_wait_bo.

Next, there are Performance Monitor (perfmon) related-ioctls: v3d_perfmon_create, v3d_perfmon_destroy, and v3d_perfmon_get_values. Performance Monitors are basically registers that are used for monitoring the performance of the V3D engine. So, tests were designed to assure that the driver was creating perfmons properly and was resilient to incorrect requests, such as trying to get a value from a non-existent perfmon.

And finally, the most interesting type of ioctls: the job submission ioctls. You can use the v3d_submit_cl ioctl to submit commands to the 3D engine, which is a tiled engine. When I think about tiled rendering, I always think about a Super Nintendo, but things can get a bit more complicated than a SNES as you can see here. The 3D engine is composed of a bin and render pipelines, each has its command list. The binning step maps the tile to a piece of the frame and the rendering step renders the tile based on its mapping.

By testing the v3d_submit_cl ioctl, it is possible to test syncing between jobs and also the V3D multisync ability.

Moreover, the V3D has also a TFU (texture formatting unit), and a CSD (compute shader dispatch), which has their ioctls: v3d_submit_tfu and v3d_submit_csd. The TFU makes format conversions and generated mipmaps and the CSD is responsible for dispatching a compute shader.

So, the idea is to write tests for all those functionalities from V3D, increasing the testability of V3D on IGT. Although things are not yet fully-done, I’ve been enjoying and working exploring the V3D, IGT, and Mesa. After this experience with Mesa and also XDC, I became more and more interested in Mesa.

A Noop Job…

In order to test the v3d_submit_cl ioctl, it was needed to design a job to be submitted. So, Melissa suggested using Mesa’s noop job specification on IGT to perform the tests. The idea was quite simple: submit a noop job and create tests based on it. But, it was not that simple after all…

First, I must say that I’m mostly a kernel developer, so I was not familiar with Mesa. So, maybe it was not that hard to figure out, but I took a while to understand Mesa’s packet and how to submit them.

The main problem I faced on submitting a noop job on IGT was: I would copy many and many Mesa files to IGT. And I took a while fighting against this idea, looking for other ways to submit a job to V3D. But, as some experience developers pointed out, packeting is the best option for it.

After some time, I was able to bring the Mesa structure to IGT with a minimal (although not that minimal) overhead. But, I’m still not able to run a successful noop job as the job’s fence is not being signaled by the end of the job.

Series Submitted

Although my noop job has not landed yet, so far, I was able to submit two series to IGT: one for the V3D driver and the other for the VC4 driver.

Apart from cleanups in the drivers, I added tests for the v3d_create_bo ioctl and the V3D’s and VC4’s perfmon ioctls. Moreover, as I was running the VC4 tests on the Raspberry Pi 4, I realized that most of the VC4 tests were failing on V3D, considering the VC4 doesn’t have rendering abilities on the Raspberry Pi 4. So, I also created checks to assure that the VC4 tests are not running on V3D.

Those series are being reviewed yet, but I hope to get them merged soon.

Next Steps

My biggest priority now is to run a noop job on IGT and for it, I’m currently running the CTS tests on the Raspberry Pi 4 in order to reproduce a noop job and understand why my current job is resulting in a hang. I added a couple of debug logs (aka printf) on Mesa and now I can see the contents of the BOs and the parameters of the submission. So, I hope to get a fully-working noop job now.

After I develop my fully working noop job, I will finish the v3d_wait_bo tests, so those only make sense if I submit a job and wait for a BO after it and design the v3d_submit_cl tests as well. For this last one, I hope to test the syncing functionalities of V3D especially.

Moreover, I hope to write soon a piece about cross-compiling CTS for the Raspberry Pi 4, which was a fun digression on this CE project.