HostAvailabilityMonitor/Documentation.en.md

# Documentation

## Overview

This repository is organized in a **Gitea-friendly repository structure** and contains two applications:

### 1. HostAvailabilityMonitor

The monitor checks technical reachability and writes:

- rolling file logs
- Windows Application Event Log
- email notifications for failures, recoveries, and a daily status summary
- a state file for transition detection, cooldown handling, and once-per-day summary tracking
- optional static HIP VM checks from `hip-vms.json` for network/ping and RDP/TCP 3389

### 2. BizTalkMonitorConfigGenerator

The generator reads BizTalk artifacts and creates a monitor configuration from them.

The goal is to avoid manually maintaining the target list and instead derive it directly from BizTalk.

---

## Repository structure for Gitea

```text
.
├── .editorconfig
├── .gitignore
├── Documentation.en.md
├── Dokumentation.md
├── HostAvailabilityMonitor.sln
├── README.md
├── installers/
│   ├── README.md
│   ├── powershell/
│   │   ├── Install-BizTalkMonitorConfigGenerator.ps1
│   │   ├── Install-Both.ps1
│   │   ├── Install-HostAvailabilityMonitor.ps1
│   │   ├── Uninstall-BizTalkMonitorConfigGenerator.ps1
│   │   └── Uninstall-HostAvailabilityMonitor.ps1
│   └── wix/
│       ├── BizTalkMonitorConfigGenerator.wxs
│       ├── HostAvailabilityMonitor.wxs
│       └── build-msi.ps1
├── scripts/
│   ├── build-release.ps1
│   ├── generate-monitor-config.ps1
│   ├── install-generator-on-server.ps1
│   ├── install-on-server.ps1
│   ├── package-deployment.ps1
│   └── register-event-source.ps1
└── src/
    ├── BizTalkMonitorConfigGenerator/
    │   ├── App.config
    │   ├── BizTalkMonitorConfigGenerator.csproj
    │   ├── Program.cs
    │   └── generator-settings.json
    └── HostAvailabilityMonitor/
        ├── App.config
        ├── HostAvailabilityMonitor.csproj
        ├── Program.cs
        ├── appsettings.json
        └── hip-vms.json
```

This layout is intentionally simple so it can be committed to **Gitea**, built on Windows, and deployed to BizTalk or application servers with minimal friction.

---

## Supported BizTalk sources

### Send Ports

By default, only **started send ports** are included.

Optionally, the **secondary transport** of a send port can also be included. Secondary transports using the **SMTP** adapter are intentionally excluded from monitor target generation because their BizTalk address usually represents an email recipient list rather than a network endpoint.

### Receive Locations

By default, only **enabled receive locations** are included.

---

## Endpoint normalization

The generator tries to transform BizTalk addresses into monitor-compatible endpoints.

### Directly supported

- `\\server\share`
- `file://...`
- `http://...`
- `https://...`
- `ftp://...`
- `sftp://...`
- `tcp://...`
- `icmp://...`
- `rdp://...`
- plain host / host:port values on a best-effort basis

### Special handling

- `net.tcp://host:port/...` is rewritten to `tcp://host:port/...`
- local file system paths such as `C:\Drop\In` are only included when `IncludeLocalFilePaths=true`
- `net.pipe://...` is skipped because the monitor does not support it

---

## HostAvailabilityMonitor: mail and status behavior

The monitor can send three kinds of emails:

1. **Failure email** when an endpoint changes from available to unavailable
2. **Recovery email** when an endpoint changes from unavailable back to available
3. **Daily status email** once per day at or after a configured local time, default **14:00 local server time**

The daily summary is intentionally **not** implemented as a second Windows task or a separate internal scheduler.

Instead, the behavior is:

- the normal scheduled monitor run starts as usual
- on every run, the monitor checks whether the configured local time has already been reached
- the **first run at or after that time** sends the daily summary email
- only **one** daily summary email is sent per calendar day
- the daily summary is marked as sent **only if SMTP delivery succeeded**

This keeps the solution resilient and simple for BizTalk server operations.

---

## Monitor configuration details

File: `src\HostAvailabilityMonitor\appsettings.json`

### Root fields

- `ApplicationName`: monitor name
- `EnvironmentName`: environment label such as `HIP Production`
- `Runtime`: runtime behavior
- `Logging`: file logging configuration
- `EventLog`: Windows Event Log configuration
- `Email`: SMTP and notification configuration
- `HipVirtualMachines`: optional static VM checks
- `Targets`: endpoints to probe

### Runtime section

Important fields in `Runtime`:

- `MaxConcurrency`: maximum number of parallel endpoint checks
- `DefaultValidateUncPathAccess`: default for UNC targets without their own `ValidateUncPathAccess` value
- `NonZeroExitCodeOnAnyFailure`: exit with code `2` when endpoint failures are found
- `NonZeroExitCodeOnExecutionError`: exit with code `1` on technical execution errors
- `StateDirectory`: directory for the state file
- `StateFileName`: state file name, default `monitor-state.json`

### Logging section

Important fields in `Logging`:

- `LogDirectory`: directory for rolling log files
- `FilePrefix`: prefix for daily log files
- `RetentionDays`: number of daily log files to retain, default `30`; `0` disables automatic cleanup

### Email section

Important fields in `Email`:

- `Enabled`: enable or disable email delivery completely
- `SmtpHost`, `SmtpPort`, `UseSsl`, `UseDefaultCredentials`, `Username`, `Password`
- `From`, `To`, `Cc`, `Bcc`, `SubjectPrefix`
- `SendOnFailure`: send an email on failure
- `SendOnRecovery`: send an email on recovery
- `SendDailySummary`: send a daily status summary
- `DailySummaryHourLocal`: local server hour, for example `14`
- `DailySummaryMinuteLocal`: local server minute, for example `0`
- `OnlyOnStateChange`: for failure/recovery emails, react only to state changes
- `CooldownMinutes`: cooldown for repeated failure emails when `OnlyOnStateChange=false`
- `TimeoutSeconds`: SMTP timeout

Recipient notes:

- `To`, `Cc`, and `Bcc` are JSON arrays of email addresses.
- Configure multiple recipients by adding multiple array entries.
- A single semicolon-separated string is not the supported format.
- At least one recipient across `To`, `Cc`, or `Bcc` must be configured when `Email.Enabled=true`.

### Example email block

```json
"Email": {
  "Enabled": true,
  "SmtpHost": "smtp.example.local",
  "SmtpPort": 25,
  "UseSsl": false,
  "UseDefaultCredentials": false,
  "Username": "",
  "Password": "",
  "From": "monitor@example.local",
  "To": [
    "operations@example.local",
    "integration-oncall@example.local"
  ],
  "Cc": [
    "integration-leads@example.local"
  ],
  "Bcc": [],
  "SubjectPrefix": "[HostAvailabilityMonitor]",
  "SendOnFailure": true,
  "SendOnRecovery": true,
  "SendDailySummary": true,
  "DailySummaryHourLocal": 14,
  "DailySummaryMinuteLocal": 0,
  "OnlyOnStateChange": true,
  "CooldownMinutes": 0,
  "TimeoutSeconds": 30
}
```

### Example for multiple recipients

```json
"Email": {
  "Enabled": true,
  "From": "monitor@example.local",
  "To": [
    "operations@example.local",
    "integration-oncall@example.local"
  ],
  "Cc": [
    "integration-leads@example.local"
  ],
  "Bcc": [
    "audit@example.local"
  ]
}
```

### Daily summary content

The daily status email contains:

- monitor name
- environment
- machine name
- time window from local midnight until send time
- today's successful and failed probe counts based on the current day's rolling log
- a current snapshot of the latest monitor run
- a list of currently unavailable endpoints including:
  - application
  - name
  - endpoint
  - kind
  - status
  - detail
  - host/port/HTTP status when available
- a HIP VM snapshot with network and RDP status per VM when `HipVirtualMachines.Enabled=true`

Note:

The daily counters are derived from the **current day's rolling log file**. If that evaluation fails for any reason, email delivery still continues; the counters fall back to `0` and the incident is logged.

---

## HIP VM Monitoring

The HIP VMs are kept in a dedicated file:

```text
src\HostAvailabilityMonitor\hip-vms.json
```

This keeps them independent from the `Targets` regenerated by the BizTalk generator. Changes to BizTalk send ports or receive locations do not overwrite the static VM list.

Enable the feature in `appsettings.json`:

```json
"HipVirtualMachines": {
  "Enabled": true,
  "ConfigPath": "hip-vms.json",
  "ApplicationName": "HIP Virtual Machines",
  "TimeoutSeconds": 5,
  "RdpPort": 3389,
  "CheckNetwork": true,
  "CheckRdp": true
}
```

The `hip-vms.json` file contains group, name, address, description, and enabled state for each VM. At startup, the monitor turns every enabled VM into normal monitor targets:

- `Network`: `icmp://<address>` for ping/network connectivity
- `RDP`: `rdp://<address>:3389` for TCP connectivity to the RDP service

These targets use the same state file as all other checks. Lost network or RDP reachability and later recoveries are therefore reported through the normal failure/recovery emails. File logs and Windows Event Log entries include source, group, machine, and check type.

---

## Generator configuration details

File: `generator-settings.json`

### Root fields

- `GeneratorName`: display name for generator logs and event log entries
- `EnvironmentName`: environment label such as `HIP Production`
- `Logging`: generator logging settings
- `EventLog`: generator event log settings
- `BizTalk`: discovery rules
- `Output`: output path and generation rules
- `GeneratedMonitorConfig`: template for the final monitor JSON

### BizTalk section

- `ConnectionString`: connection to BizTalkMgmtDb
- `IncludeSendPorts`: read send ports
- `IncludeReceiveLocations`: read receive locations
- `OnlyStartedSendPorts`: include active send ports only
- `OnlyEnabledReceiveLocations`: include active receive locations only
- `IncludeSecondaryTransportOnSendPorts`: include secondary send transports
- SMTP send ports and SMTP secondary transports are skipped automatically by the generator
- `IncludeDynamicSendPorts`: include dynamic send ports
- `IncludeLocalFilePaths`: include local file system paths
- `IgnoreSystemApplications`: skip system applications
- `IgnoreApplications`: ignore applications by wildcard pattern
- `IgnoreArtifactNamePatterns`: ignore ports/locations by wildcard pattern
- `ApplicationMappings`: map BizTalk application names to business application names
- `ArtifactMappings`: map specific artifacts to business application names

### Output section

- `Path`: target path of the generated monitor JSON
- `OverwriteExisting`: overwrite existing file
- `FailIfNoTargetsFound`: return exit code 2 when no targets were found
- `TargetTimeoutSeconds`: default timeout for each generated target
- `TargetEnabled`: default `Enabled` value for each generated target
- `DefaultHttpMethod`: default HTTP method for generated HTTP/HTTPS targets

### GeneratedMonitorConfig section

This is the template for the monitor runtime configuration.

It contains:

- monitor `ApplicationName`
- `EnvironmentName`
- `Runtime`
- `Logging`
- `EventLog`
- `Email`
- `HipVirtualMachines`

`Targets` is rebuilt by the generator at runtime.

Important:

When the generator writes the monitor JSON, runtime, logging, daily summary, and static HIP VM settings are also copied into the generated target file. That means values such as `GeneratedMonitorConfig.Runtime.StateFileName`, `GeneratedMonitorConfig.Logging.RetentionDays`, `GeneratedMonitorConfig.Email.SendDailySummary`, `DailySummaryHourLocal`, `DailySummaryMinuteLocal`, and `GeneratedMonitorConfig.HipVirtualMachines` flow directly into the output monitor configuration.

---

## How to use the generator

### Option A: run the EXE directly

```powershell
C:\Tools\BizTalkMonitorConfigGenerator\BizTalkMonitorConfigGenerator.exe C:\Tools\BizTalkMonitorConfigGenerator\generator-settings.json
```

### Option B: run the wrapper script

```powershell
.\scripts\generate-monitor-config.ps1 -InstallPath C:\Tools\BizTalkMonitorConfigGenerator
```

### Expected result

After the run, you should have:

- a generated JSON file, for example `C:\Tools\BizTalkMonitorConfigGenerator\generated-monitor-appsettings.json`
- a generator log file under `logs\`
- event log entries in the Windows `Application` log

### Run the monitor afterwards

```powershell
C:\Tools\HostAvailabilityMonitor\HostAvailabilityMonitor.exe C:\Tools\BizTalkMonitorConfigGenerator\generated-monitor-appsettings.json
```

---

## Example flow on a BizTalk server

### 1. Install the generator

```powershell
.\scripts\install-generator-on-server.ps1 `
  -SourcePath .\artifacts\publish\net472\BizTalkMonitorConfigGenerator `
  -InstallPath C:\Tools\BizTalkMonitorConfigGenerator `
  -RegisterEventSource
```

### 2. Adjust the generator settings

Edit this file:

```text
C:\Tools\BizTalkMonitorConfigGenerator\generator-settings.json
```

Typical values:

- `EnvironmentName = "HIP Production"`
- `Output.Path = "C:\\Tools\\BizTalkMonitorConfigGenerator\\generated-monitor-appsettings.json"`
- `BizTalk.ConnectionString = "SERVER=.;DATABASE=BizTalkMgmtDb;Integrated Security=SSPI"`
- `GeneratedMonitorConfig.Email.SendDailySummary = true`
- `GeneratedMonitorConfig.Email.DailySummaryHourLocal = 14`
- `GeneratedMonitorConfig.Email.DailySummaryMinuteLocal = 0`

### 3. Run the generator

```powershell
C:\Tools\BizTalkMonitorConfigGenerator\BizTalkMonitorConfigGenerator.exe C:\Tools\BizTalkMonitorConfigGenerator\generator-settings.json
```

### 4. Check the generated JSON

Confirm that the file was created and contains `Targets`.

Also verify:

- the generated JSON contains `SendDailySummary`
- the daily summary time values are correct

### 5. Install the monitor

```powershell
.\scripts\install-on-server.ps1 `
  -SourcePath .\artifacts\publish\net472\HostAvailabilityMonitor `
  -InstallPath C:\Tools\HostAvailabilityMonitor `
  -RegisterEventSource
```

### 6. Start the monitor with the generated JSON

```powershell
C:\Tools\HostAvailabilityMonitor\HostAvailabilityMonitor.exe C:\Tools\BizTalkMonitorConfigGenerator\generated-monitor-appsettings.json
```

### 7. Configure Task Scheduler

Recommended setup:

- generator every 30 minutes
- monitor every 30 minutes, 1 to 2 minutes after the generator
- ensure there is at least one monitor run **after 14:00 local time** so the daily summary can be sent

---

## Generated target names

To make logs and alert emails easy to understand, the generator creates descriptive names such as:

- `SendPort: SAP_Outbound (Primary) [WCF-SQL]`
- `SendPort: PartnerA_SFTP (Secondary) [SFTP]`
- `ReceiveLocation: InboundOrders / RL_FileDrop [FILE]`

The business `ApplicationName` of the generated target is resolved in this order:

1. `ArtifactMappings`
2. `ApplicationMappings`
3. BizTalk application name as fallback

---

## Generator logging and resilience

The generator is designed defensively:

- daily rolling logs
- retries when appending to the log file
- event log is best effort only
- atomic output file writing (`.tmp` + rename)
- existing target file is backed up as `.bak` before overwrite
- unsupported endpoints are logged and skipped
- discovery continues even if individual artifacts are unusable

---

## Monitor logging and resilience

The monitor is also defensive:

- daily rolling logs
- default retention of 30 days, configurable through `Logging.RetentionDays`
- event log writes are best effort only
- state file is written atomically
- failure/recovery emails follow explicit notification rules
- daily summary is sent only once per day
- the summary is marked as sent only after successful SMTP delivery
- targets that look like SMTP/email recipient lists are handled defensively as `Skipped` and do not raise DOWN alerts
- log parsing or SMTP errors do not crash the monitor run hard

---

## Server installation quick guide

### Create a deployment package

```powershell
.\scripts\build-release.ps1 -CreateDeploymentPackage
```

or separately:

```powershell
.\scripts\package-deployment.ps1
```

The result is a ZIP under `artifacts\deploy\net472\`.

### PowerShell installers (recommended)

Install both applications from the deployment ZIP:

```powershell
.\installers\powershell\Install-Both.ps1 `
  -PackageRoot . `
  -BaseInstallPath "C:\Program Files\JR IT Services\BizTalk Endpoint Monitor" `
  -BaseConfigPath "C:\ProgramData\JR IT Services\BizTalk Endpoint Monitor" `
  -RegisterEventSources
```

Install monitor only:

```powershell
.\installers\powershell\Install-HostAvailabilityMonitor.ps1 `
  -PackageRoot .
```

Install generator only:

```powershell
.\installers\powershell\Install-BizTalkMonitorConfigGenerator.ps1 `
  -PackageRoot .
```

### WiX MSI sources (optional)

WiX source files are included for both applications. On a Windows build machine with WiX Toolset v3 you can generate MSI packages:

```powershell
.\installers\wix\build-msi.ps1 `
  -PublishRoot .\artifacts\publish\net472 `
  -OutputRoot .\artifacts\msi
```

### Individual installation using the original scripts

#### Monitor

```powershell
.\scripts\install-on-server.ps1 `
  -SourcePath .\artifacts\publish\net472\HostAvailabilityMonitor `
  -InstallPath C:\Tools\HostAvailabilityMonitor `
  -RegisterEventSource
```

#### Generator

```powershell
.\scripts\install-generator-on-server.ps1 `
  -SourcePath .\artifacts\publish\net472\BizTalkMonitorConfigGenerator `
  -InstallPath C:\Tools\BizTalkMonitorConfigGenerator `
  -RegisterEventSource
```

Then typically:

1. adjust `generator-settings.json`
2. run the generator
3. verify the generated monitor JSON
4. run the monitor against that JSON
5. configure Task Scheduler

---

## Build note

In this environment, no real Windows build against .NET Framework 4.7.2 and BizTalk ExplorerOM could be executed. The same applies to real WiX-built MSI binaries because no Windows/WiX toolchain is available here. The code and repository structure were updated, but the first real build and validation must happen on a Windows/BizTalk system.