🔥Overview
As a DevOps engineer, one of the critical aspects of maintaining a secure and reliable system is effective patch management. In this blog post, we'll delve into what patch management is, why it's essential, and how the process of patching and rolling back works in Linux.
👍Patch Management: An Overview
Patch management refers to the process of keeping a computer system up-to-date with the latest software updates, fixes, and security patches. These updates are released by software vendors, including the Linux community, to address various issues such as security vulnerabilities, bugs, performance improvements, and feature enhancements.
😁Proper patch management is crucial for several reasons:
Security: Unpatched systems are vulnerable to various cyber threats. Regularly applying security patches helps protect your Linux system from potential exploits and attacks.
Stability: Patches not only address security concerns but also fix bugs and improve overall system stability, leading to a more reliable computing environment.
Performance: Some patches can optimize system performance, ensuring your Linux system runs efficiently.
Compatibility: Staying updated with patches ensures compatibility with newer software and hardware releases.
The Patching Process in Linux:
The patching process in Linux typically involves the following steps:
Monitoring: You need to stay informed about new updates and patches. Monitoring official channels, mailing lists, security advisories, and release notes from Linux distributions and software vendors is crucial.
Assessment: Before applying a patch, it's essential to assess its impact on the system. Some patches may introduce compatibility issues or conflicts with existing configurations. Having a test environment to evaluate patches before deploying them in production is a good practice.
Backup: Before applying any patches, it is essential to back up critical data and configurations. This step acts as a safety net in case anything goes wrong during the patching process.
Applying Patches: Linux distributions usually come with package managers (e.g., apt for Debian/Ubuntu, yum/dnf for Red Hat/CentOS/Fedora) that simplify the process of downloading and installing patches. By running specific commands, you can update individual packages or the entire system.
😁PATCH MANAGEMENT:
Pre-requisites:
Inform the application owners and stakeholders about the updates and changes.
Send a list of all the apps and kernel upgrades
Check the module files for pre-checks - ls -l /var/lib/ modules/kernel-versions/mod.dep
To check the list of apps and kernel versions
😎Post checks:
Check the mount points if there is an error.
Check if the panic attack errors for kernel
Check for the latest versions of kernel
During OS patching, you check mostly the kernel and host file entries
lsblk lists information about all available or specified block devices. The lsblk command reads the sysfs filesystem and udev db to gather information. If the udev database is not available or lsblk is compiled without udev support, then it tries to read LABELs, UUIDs, and filesystem types from the block device.
After post-validation checking all these, we forward the server to the database and application teams to check.
They will validate from there end and perform the sanity checks (functuality checks, whether their apps are running smoothly or not, and login checks).
Lastly, they will check and confirm that the update was successful.
Roll back patches on Linux Server with Transaction ID:
Command yum history list all gives all the ID's.
You cannot roll back to the previous version of the kernel.In case you want the old kernel, we need to interrupt the GRUB and then start the process.
Roll back the previous versions using the command:-yum history undo 10 (Transaction ID)
The downgrade of the packages is visible in the screenshot
After every rollback, you need to reboot -f.
Check the kernel (glibc) version Previous and current versions can be seen
Once done, check with the application team for sanity checks.
🥲Troubleshooting:
Issues after Patching Activity(Recreate kernel panic issue, then try to fix it.)
After a post-patch, sometimes the drive or image of the kernel gets corrupted.
Boot process BIOS first Initramfs runs in GRUB file then kernel boots up and systemd service runs all the processes
You change the priority of the boot menu to CD-ROM and save changes
Go to rescue mode and need to recreate the file
Changing the root fs to different mount points
Dracut command to recreate the initramfs image and file
Once the image is created, reboot and change the priority to hard drive in boot.
To check on the patching management in detail, please check the below video
😊Conclusion:
Patch management is a fundamental responsibility of Linux administrators to ensure the security, stability, and performance of their systems. By staying informed about updates, carefully assessing patches, and having a rollback plan in place, administrators can effectively manage patches and maintain a reliable and secure Linux infrastructure. Remember, always test patches in a controlled environment before applying them in a production environment, and don't forget to back up your critical data regularly. Happy patching!