Update Fix FIP-FS Handling Error Y2K22 Email Stuck in Transport Queues When Updating Exchange Server Antimalware

phuong.nguyen 2022/01/04 - 10:36 PM

Contents

1 Description
2 Cause
3 Solution
- 3.1 Method 1: Using the Automated Solution
- 3.2 Method 2: Update manual

Description

According to the article on 01/01/2022, there is an important error message about the inter-Exchange failure to send/receive Y2K22 here

Recently, after reporting on January 2, 2022, brothers cried around the world because of an error related to Exchange 2016 products, Exchange 2019. Microsoft has confirmed that this is a Y2K22 error.

Microsoft

Microsoft has created a workaround to resolve the issue of messages getting stuck in the transport queue on Exchange Server 2016 and Exchange Server 2019 due to a potential date issue in the signature file used by the malware scanning engine in Exchange Server. .

Log Name: Application 
Source: FIPFS 
Logged: 1/1/2022 1:03:42 AM 
Event ID: 5300 
Level: Error 
Computer: NVPSRVEX01.viettechgroup.lab
Description: The FIP-FS "Microsoft" Scan Engine failed to load. PID: 23092, Error Code: 0x80004005. Error Description: Can't convert "2201010001" to long.

Log Name: Application 
Source: FIPFS 
Logged: 1/1/2022 11:47:16 AM 
Event ID: 1106 
Level: Error 
Computer: NVPSRVEX01.viettechgroup.lab
Description: The FIP-FS Scan Process failed initialization. Error: 0x80004005. Error Details: Unspecified error.

Cause

Security researcher and Exchange admin Joseph Roosen said that this is caused by Microsoft using a signed int32 variable to store the value of a date, which has a maximum value of 2,147,483,647.

However, dates in 2022 have a minimum value of 2,201,010,001 or larger, which is greater than the maximum value that can be stored in the signed int32 variable, causing the scanning engine to fail and not release mail for delivery.

Solution

Method 1: Using the Automated Solution

Microsoft has updated an automatic script ResetscanengineVersion. Note how many Exchange servers in the system, run all of them to update.

Download Scripts below:

https://aka.ms/ResetScanEngineVersion

https://viettechgroup.vn/upload/Reset-ScanEngineVersion.ps1

Perform the update as follows, if you have disabled or bypassed antimalware scanning, you must re-enable it before doing this Update operation.

Reducing Memory Consumption of the Exchange Online…

How to manually clean up a Distributed Virtual Switch (VDS)…

Released: October 2023 Exchange Server Security Updates

Run EMS Administrators on all Exchange Servers we have.

.\Reset-ScanEngineVersion.ps1

[PS] D:\Y2K22>.\Reset-ScanEngineVersion.ps1
NVPSRVEX01 Stopping services...
WARNING: Waiting for service 'Microsoft Filtering Management Service (FMS)' to stop...
WARNING: Waiting for service 'Microsoft Filtering Management Service (FMS)' to stop...
WARNING: Waiting for service 'Microsoft Filtering Management Service (FMS)' to stop...
WARNING: Waiting for service 'Microsoft Filtering Management Service (FMS)' to stop...
WARNING: Waiting for service 'Microsoft Filtering Management Service (FMS)' to stop...
WARNING: Waiting for service 'Microsoft Filtering Management Service (FMS)' to stop...
WARNING: Waiting for service 'Microsoft Filtering Management Service (FMS)' to stop...
WARNING: Waiting for service 'Microsoft Filtering Management Service (FMS)' to stop...
WARNING: Waiting for service 'Microsoft Filtering Management Service (FMS)' to stop...
WARNING: Waiting for service 'Microsoft Filtering Management Service (FMS)' to stop...
WARNING: Waiting for service 'Microsoft Filtering Management Service (FMS)' to stop...
WARNING: Waiting for service 'Microsoft Filtering Management Service (FMS)' to stop...
WARNING: Waiting for service 'Microsoft Filtering Management Service (FMS)' to stop...
WARNING: Waiting for service 'Microsoft Filtering Management Service (FMS)' to stop...
WARNING: Waiting for service 'Microsoft Filtering Management Service (FMS)' to stop...
WARNING: Waiting for service 'Microsoft Filtering Management Service (FMS)' to stop...
WARNING: Waiting for service 'Microsoft Filtering Management Service (FMS)' to stop...
WARNING: Waiting for service 'Microsoft Filtering Management Service (FMS)' to stop...
WARNING: Waiting for service 'Microsoft Filtering Management Service (FMS)' to stop...
WARNING: Waiting for service 'Microsoft Filtering Management Service (FMS)' to stop...
WARNING: Waiting for service 'Microsoft Filtering Management Service (FMS)' to stop...
WARNING: Waiting for service 'Microsoft Filtering Management Service (FMS)' to stop...
WARNING: Waiting for service 'Microsoft Filtering Management Service (FMS)' to stop...
WARNING: Waiting for service 'Microsoft Filtering Management Service (FMS)' to stop...
WARNING: Waiting for service 'Microsoft Filtering Management Service (FMS)' to stop...
WARNING: Waiting for service 'Microsoft Filtering Management Service (FMS)' to stop...
WARNING: Waiting for service 'Microsoft Filtering Management Service (FMS)' to stop...
WARNING: Waiting for service 'Microsoft Filtering Management Service (FMS)' to stop...
WARNING: Waiting for service 'Microsoft Filtering Management Service (FMS)' to stop...
WARNING: Waiting for service 'Microsoft Filtering Management Service (FMS)' to stop...
WARNING: Waiting for service 'Microsoft Filtering Management Service (FMS)' to stop...
WARNING: Waiting for service 'Microsoft Filtering Management Service (FMS)' to stop...
WARNING: Waiting for service 'Microsoft Filtering Management Service (FMS)' to stop...
WARNING: Waiting for service 'Microsoft Filtering Management Service (FMS)' to stop...
WARNING: Waiting for service 'Microsoft Filtering Management Service (FMS)' to stop...
WARNING: Waiting for service 'Microsoft Filtering Management Service (FMS)' to stop...
WARNING: Waiting for service 'Microsoft Filtering Management Service (FMS)' to stop...
WARNING: Waiting for service 'Microsoft Filtering Management Service (FMS)' to stop...
WARNING: Waiting for service 'Microsoft Filtering Management Service (FMS)' to stop...
WARNING: Waiting for service 'Microsoft Filtering Management Service (FMS)' to stop...
WARNING: Waiting for service 'Microsoft Filtering Management Service (FMS)' to stop...
WARNING: Waiting for service 'Microsoft Filtering Management Service (FMS)' to stop...
WARNING: Waiting for service 'Microsoft Filtering Management Service (FMS)' to stop...
WARNING: Waiting for service 'Microsoft Filtering Management Service (FMS)' to stop...
WARNING: Waiting for service 'Microsoft Filtering Management Service (FMS)' to stop...
WARNING: Waiting for service 'Microsoft Filtering Management Service (FMS)' to stop...
WARNING: Waiting for service 'Microsoft Filtering Management Service (FMS)' to stop...
WARNING: Waiting for service 'Microsoft Filtering Management Service (FMS)' to stop...
WARNING: Waiting for service 'Microsoft Filtering Management Service (FMS)' to stop...
NVPSRVEX01 Removing Microsoft engine folder...
NVPSRVEX01 Emptying metadata folder...
NVPSRVEX01 Starting services...
WARNING: Waiting for service 'Microsoft Filtering Management Service (FMS)' to start...
WARNING: Waiting for service 'Microsoft Filtering Management Service (FMS)' to start...
WARNING: Waiting for service 'Microsoft Filtering Management Service (FMS)' to start...
WARNING: Waiting for service 'Microsoft Filtering Management Service (FMS)' to start...
WARNING: Waiting for service 'Microsoft Filtering Management Service (FMS)' to start...
WARNING: Waiting for service 'Microsoft Filtering Management Service (FMS)' to start...
WARNING: Waiting for service 'Microsoft Filtering Management Service (FMS)' to start...
WARNING: Waiting for service 'Microsoft Exchange Transport (MSExchangeTransport)' to start...
WARNING: Waiting for service 'Microsoft Exchange Transport (MSExchangeTransport)' to start...
WARNING: Waiting for service 'Microsoft Exchange Transport (MSExchangeTransport)' to start...
NVPSRVEX01 Starting engine update...
Running as VIETTECHGROUP\Administrator.
--------
Connecting to NVPSRVEX01.viettechgroup.lab.
Dispatched remote command. Start-EngineUpdate -UpdatePath http://amupdatedl.microsoft.com/server/amupdate

The update process takes place in 15-30 minutes depending on the network system. By checking with this command:

Add-PSSnapin Microsoft.Forefront.Filtering.Management.Powershell.
Get-EngineUpdateInformation

Make sure the version is 2112330001 and check in the event viewer that there are no errors

Method 2: Update manual

Remove existing engine and metadata
1. Stop the Microsoft Filtering Management service. When prompted to also stop the Microsoft Exchange Transport service, click Yes.
2. Use Task Manager to ensure that updateservice.exe is not running.
3. Delete the following folder: %ProgramFiles%\Microsoft\Exchange Server\V15\FIP-FS\Data\Engines\amd64\Microsoft.
4. Remove all files from the following folder: %ProgramFiles%\Microsoft\Exchange Server\V15\FIP-FS\Data\Engines\metadata.

Update to latest engine
1. Start the Microsoft Filtering Management service and the Microsoft Exchange Transport service.
2. Open the Exchange Management Shell, navigate to the Scripts folder (%ProgramFiles%\Microsoft\Exchange Server\V15\Scripts), and run Update-MalwareFilteringServer.ps1 <server FQDN>.

Verify engine update info
1. In the Exchange Management Shell, run Add-PSSnapin Microsoft.Forefront.Filtering.Management.Powershell.
2. Run Get-EngineUpdateInformation and verify the UpdateVersion information is 2112330001.

After updating the engine, we also recommend that you verify that mail flow is working and that FIPFS error events are not present in the Application event log.

Good luck to you

Source

Microsoft teams Exchange: https://techcommunity.microsoft.com/t5/exchange-team-blog/email-stuck-in-transport-queues/ba-p/3049447

Phương Nguyễn : http://phuongnguyenit.com/cap-nhat-ban-xu-ly-fip-fs-loi-lien-quan-y2k22-treo-email-queues-khi-update-antimalware/

Y2k22 Exchange server