This is a weekly report from the I&R (Infrastructure & Release Engineering) Team. We provide you both infographic and text version of the weekly report. If you just want to quickly look at what we did, just look at the infographic. If you are interested in more in depth details look below the infographic.
Week: 24 Mar – 28 Mar 2025

Infrastructure & Release Engineering
The purpose of this team is to take care of day to day business regarding CentOS and Fedora Infrastructure and Fedora release engineering work.
It’s responsible for services running in Fedora and CentOS infrastructure and preparing things for the new Fedora release (mirrors, mass branching, new namespaces etc.).
List of planned/in-progress issues
Fedora Infra
- In progress:
- Planned Outage – update/reboots – 2025-03-26 21:00 UTC
- Disable self-serve project creation on pagure.io
- OTP locked me out of my account
- Link in staging distgit instance leads to prod auth
- re-add datagrepper nagios checks (and add to zabbix?)
- Fix logrotate on kojipkgs01/02
- 2025 datacenter move (IAD2->RDU3)
- Broken link for STG IPA CA certificate, needed for staging CentOS Koji cert
- [CommOps] Open Data Hub on Communishift
- retire easyfix
- Add StartInstanceRefresh permission for fedora-ci-testing-farm user in AWS
- Deploy Element Server Suite operator in staging
- Pagure returns error 500 trying to open a PR on https://src.fedoraproject.org/rpms/python-setuptools-gettext
- Create a POC integration in Konflux for fedora-infra/webhook-to-fedora-messaging
- maubot-meetings bot multi line paste is cut
- setup ipa02.stg and ipa03.stg again as replicas
- Manage our new testing.farm domain via AWS Route53
- Add support for creating RDS instances under
testing-farm-
prefix - Move OpenShift apps from deploymentconfig to deployment
- The process to update the OpenH264 repos is broken
- httpd 2.4.61 causing issue in fedora infrastructure
- Support allocation dedicated hosts for Testing Farm
- EPEL minor version archive repos in MirrorManager
- vmhost-x86-copr01.rdu-cc.fedoraproject.org DOWN
- Add yselkowitz to list to notify when ELN builds fail
- Cleaning script for communishift
- move resultsdb-ci-listener deployment pipeline
- Setup RISC-V builder(s) VM in Fedora Infrastructure
- Move from iptables to firewalld
- Help me move my discourse bots to production?
- rhel9 adoption
- Replace Nagios with Zabbix in Fedora Infrastructure
- Migration of registry.fedoraproject.org to quay.io
- Done:
- I try to run the Fedora Wiki localy with vagrant
- Please add sponsors to the @vespa project.
- Spam comment from new user account
- Change kde.fedoraproject.org to point to fedoraproject.org/kde/
- join.fp.o is a broken redirect
- Blocked access to Fedora repositories
- Network connectivity issues on koji builders (partial downloads, etc.)
- Google Cloud Engine credentials for fedora-image-uploader
CentOS Infra including CentOS CI
- In progress:
- Add kiwi to CentOS Stream Koji
- Replace some Infrastructure servers (out of warranty)
- Increase Storage capacity for koji built artifacts
- Testing Jenkins – c10s x86_64 metal doesn’t provision
- Release Improvements
- hardware raid controller issue on sponsored node
- Ensuring ansible automation in place can manage/control el10 nodes
- Retire Public CI SIG
- [spike] : investigating ppc64le kvm host option with el9/el10 (for Power10)
- [spike] : investigating needed deps (poetry) for duffy on el9
- Onboarding new infra SIG member
- Migrate main backup storage pool to different hardware
- [spike] : investigating options/alternatives for the upcoming DC move
- Done:
- New tags needed for Sama v4.22 in Storage SIG for CentOS Stream 9
- Drop options from virt9s-ovirt-45-el9s-build
- Please update rpm-specs-latest.tar.xz
- New Mirror
- Creating a prometheus resource in cloud-softwarefactory namespace hangs
- Setup for the new Proposed Updates SIG
- Making CentOS Stream CoreOS available
Release Engineering
- In progress:
- Accidentally created a branch in
rust-tonic-types
- Mass retirement of packages with uninitialized rawhide branch
- Please send openh264-2.6.0 to Cisco
- 300+ F42FTBFS bugzillas block the F41FTBFS tracker
- Packages that have not been rebuilt in a while or ever
- Send compose reports to a to-be-created separate ML
- .sqlite metadata missing in f41-updates and f41-updates-testing repositories
- Could we have fedoraproject-updates-archive.fedoraproject.org for Rawhide?
- Investigate and untag packages that failed gating but were merged in via mass rebuild
- a few mass rebuild bumps failed to git push – script should retry or error
- Package retirements are broken in rawhide
- Implement checks on package retirements
- Untag containers-common-0.57.1-6.fc40
- Provide stable names for images
- Packages that fail to build SRPM are not reported during the mass rebuild bugzillas
- When orphaning packages, keep the original owner as co-maintainer
- Create an ansible playbook to do the mass-branching
- RFE: Integration of Anitya to Packager Workflow
- Fix tokens for ftbfs_weekly_reminder. script
- Update bootloader components assignee to “Bootloader Engineering Team”for Improved collaboration
- Accidentally created a branch in
- Done:
- Stalled EPEL package: python-slugify
- Stalled EPEL package request: python-rfc3986-validator
- Stalled EPEL package request: python-w3lib
- Stalled EPEL package request: python-uvloop & python-fqdn
- Stalled EPEL package request: python-json-logger
- Stalled EPEL package request: python-strict-rfc3339
- stalled EPEL request: python-aiounittest
- Unretire python-eventlet
- Stalled EPEL package: python-numcodecs
- Unretire screengrab
- Stalled EPEL package: python-httptools
- Unretirement request: mactel-boot
- Unable to create update from side tag
- Create detached signatures for the ignition 2.21.0 release
- untag cmake-4.x.x builds from f43/rawhide
- Unretirement request: rust-elasticlunr-rs
- Unorphan workrave
- Please install isomd5sum package on secondary01.fedoraproject.org
- Unretire datanommer-commands and python-datanommer-models packages
- Create
image-builder-build
koji build group
If you have any questions or feedback, please respond to this report or contact us on #redhat-cpe channel on matrix.
The post Infra and RelEng Update – Week 13 2025 appeared first on Fedora Community Blog.