Software updates are vital for IoT device management. When operating a fleet of thousands and even hundreds of thousands of devices it is important to be able to monitor the software update process. Understanding which groups of devices have successfully completed the update and which ones encountered a problem is important both for statistics and for debugging reasons. Bosch IoT Device Management provides these important capabilities through its user interfaces.
Bosch IoT Device Management provides capabilities to trace and monitor how the update process is going through. Although the actual monitoring of executions is handled through the mass management capabilities of the device management layer (Bosch IoT Manager), all service layers are involved in the process.
Assigning a distribution set to a device or executing a rollout to a fleet of devices in the Software update layer (Bosch IoT Rollouts) will also communicate this assignment to the Device management layer (Bosch IoT Manager).
As part of this communication a mass management rule is created. Its Groovy script includes the install command to the device(s).
Generally a rollout with 10 000 devices creates 10 rules with 1000 devices. Of course this may differ when using rollout groups during the creation of the rollout.
The Software update layer generates the needed Groovy scripts, creates, enables and fires the rule with a manual trigger on all devices that are online. Basically one software update assignment in IoT Rollouts leads to many operations in IoT Manager - then any user can see the detailed flow of the process.
The install command is sent through the Digital twin layer to the Device communication layer and then to the actual device.
When the device receives the command, the Device communication layer returns a confirmation that the command is successfully received.
Within the rules perspective you can monitor what is happening with the rule execution i.e. see the related task executions. Within the task executions you can monitor their partial executions and see the details about the communications with the device. In this particular case - whether the device has already received the update or not. In case of errors during the device communication, messages will be visible on this layer. Bosch IoT Manager waits for the install command to be delivered to the device i.e. the device communication layer confirms that the command is received until it finishes the execution successfully.
Once the download of the artifact and actual installation on the device is complete - the Device connectivity layer sends a confirmation of completed installation.
The update process is reported via the SoftwareUpdatable feature through the Digital twin layer.
The Software update layer (Bosch IoT Rollouts) does not keep track of the status of the task executions in IoT Manager, but rather waits for the device to report the state of the installation. IoT Rollouts waits for the install command to be completed i.e. the Digital twin layer reports that the update has been successful until Bosch IoT Rollouts marks the corresponding action as completed.
What happens when a device is offline?
There is a particular logic implemented in cases when a device is offline during the first attempt for software update.
Once the rule has been manually triggered, the device has 60 sec to acknowledge the install command. If the acknowledgement does not happen - the execution fails with timeout error, which is communicated from the Digital twin layer to the Device management layer.
All created rules have an event based trigger to trigger the execution again in cases when the device is offline. Once the device goes online, the Device connectivity layer emits a DeviceOnlineEvent, and the event based trigger which is listening for a DeviceOnlineEvent will trigger also the rule i.e. will send the install command again for this particular device.
See the flow diagram below for better understanding:
How does software update cancellation work?
When a software update is cancelled from IoT Rollouts, the execution for this particular device within the rule responsible for sending the install command will be cancelled.
After that, depending on which SoftwareUpdatable version you are using, there are two approaches to the cancellation:
When using SoftwareUpdatable v1.0.0 cancelling a software update is not communicated to the device, and will directly fail the rule and set it to error state in both Bosch IoT Manager and IoT Rollouts.
When using SoftwareUpdatable v2.0.0 cancelling a software update will create a new rule with specific groovy script which sends a cancel command to the device for the particular action and waits for an acknowledgment.
How many times is the install command sent to the device?
Generally speaking, the command is sent over and over again until Bosch IoT Device Management receives an acknowledgment for it by the device. If an acknowledgment is not received within 60 sec, the rule execution fails due to timeout and the rule will be triggered again upon receiving a DeviceOnline event. Every time the device comes online, the install command will be sent again in an infinite loop.
Note that you can send the install command again anytime by firing the rule manually.
This infinite loop can be stopped in two ways:
Cancelling the software update
Assigning a new distribution to the device
When are rules cleaned?
Rules are deleted based on the IoT Rollouts completion status i.e. after the update has completed successfully, not after the command has been received.
In case there are still some errors and the installation is not successful, the rule is not cleaned up, because it can be used for debugging purposes.
Monitor the software update process
You can start your monitoring activities at the Bosch IoT Rollouts UI.
Just select the target and then click the expand button (highlighted below) on the right Action history view.
Here you can see the ruleId, when has been the rule enabled and fired, when did the download of the artifact start and when did the installation finish.
The information about the start and finish of the download is reported by the device by updating the SoftwareUpdatable feature. In case the device does not provide this info, it will not be shown here.
However if you work with a big fleet of devices and need some more detailed information Bosch IoT Device Management provides two other user interfaces. They allow you to monitor in depth the communication with devices, the task executions, part of the software update rule and any error messages.
Monitoring the executions via the Developer Console
Enter the Bosch IoT Suite UI by clicking Go to Developer Console.
In case it is not there, use the search for Services and features, and click the star icon to pin it to your navigation bar.
Find and select the rule of interest.
On the right side you see the Rule properties, Scope, Execution options and the Last execution on devices.
As part of the Last execution on devices you can see statistics about the number of devices that are involved in the rule, still running, finished with success, error or warning, if executions were cancelled and how many times has the rule been triggered.
Clicking on any of the blue (hyperlinked) criteria e.g. All involved or Running, will lead you to the Executions blade of the selected group of devices:
Monitoring the executions via the Bosch IoT Manager UI
Enter the Bosch IoT Manager UI by clicking Go to Dashboard. There:
Select the Rule Management perspective.
For your convenience, you may use the Quick Views panel on the left. If the state of the rule is unknown - select All Rules.
Select the rule of interest.
In the Rule Details view you can see statistics about the number of devices that are involved in the rule, still running, finished with success, error or warning, if executions were cancelled and how many times has the rule been triggered.
To enter the task executions view click on Tasks at the bottom.
In the tasks view you can see the partial executions of this particular task.
If you have a huge list of rules, please find the one of interest by getting the ruleId from the Bosch IoT Rollouts UI (Action History) and make a search query in the other UIs.