Skip to main content

C2C Device Certification Process

Updated over 3 months ago

Overview

This article describes the minimum set of tests used to verify that a device is fully supported by Arcules Camera-to-Cloud. This article focuses on testing new models and does not test every aspect of the software.

Examples of tests not included in this document:

  • Arcules system behavior with any device that does not meet the documented Specifications.

  • Performing a factory reset on the device before completing device onboarding.

  • Entering invalid input or canceling a dialog.

Specifications

Devices must meet all of the following requirements:

  • System-on-chip: ARTPEC-8

  • Single sensor

  • Memory (minimum): 1 GB (RAM).

  • Supports 1920x1080 resolution at 10 Frames per Second (FPS)

  • Axis Object Analytics

  • Supported Firmware: Visit the Firmware Compatibility for C2C Cameras article

  • SD Card Installed (link).

  • Everything in the Arcules Network Readiness Guide (link).

Tests Coverage

This section lists the major areas to test and provide a simple checklist for reference. The entire test suite should take between 2-4 hours per device, excluding the Long-Running tests. Each major test area has a dedicated section in this document:

  1. Device Management

  2. Settings

  3. Device Drawer Tabs

  4. Long-Running

Device Management

This section focuses on testing addition and removal of devices in the Arcules Portal. Testing this section should take less than one hour and will depend on the size of the SD card installed in the device.

Add Device

This test confirms that adding a device to the Arcules Portal works as expected. The expected time for this test is based largely on the size of the SD card installed in the device under test. The table below serves as a general guide for AXIS SD cards. Actual formatting times may vary.

SD Card Size

Time to Format SD Card

Time to Appear ONLINE

128 GB or less

< 5 minutes

< 10 minutes

256 GB

5 minutes

10 minutes

512 GB

10 minutes

15 minutes

1024 GB

20-25 minutes

30 minutes

Test Procedure

1. Follow the instructions here to add the device to the Arcules Portal

Criteria for success:

  • The device should appear in the All Devices portal page within 2 minutes of starting the Arcules Orchestration ACAP and claiming the device via the portal.

  • The device will not be ONLINE immediately (see table above).

  • After the device shows as ONLINE:

    • There are no errors in the Arcules Orchestration ACAP log with the following exceptions:

      • If the status page is opened before the device is claimed, there may be an error:

        • Error getting camera status: failed to get device name

      • If the device is not claimed immediately in the portal, there may be several errors:

        • failed to get device claim error

      • Updates to Arcules Orchestration can trigger numerous errors mentioning that servers are shutting down.

    • Clicking on the device in the table shows a page called the Device Drawer with the following:

      • Name of the device.

      • Settings button.

      • X button to close the Device Drawer.

      • ONLINE status.

      • Overview, Storage, and Analytics tabs.

Remove Device

This test confirms that deleting a device from the Arcules Portal works as expected. This test should take about five minutes.

Test Procedure

  1. Press the Remove Device button at the bottom of the Overview tab of the Device Drawer.

  2. Check the box on the resulting dialog.

  3. Press Delete.

Criteria for Success

  • The portal indicates that the device is being removed.

  • The Device Drawer is closed.

  • The device initiates a factory reset within 2 minutes.

    • After the factory reset completes, navigating to the device in a web browser should show an initial setup page similar to the following:

Settings

This section focuses on testing the functionality for all modal settings dialogs for the device. This section assumes that the device has already been added to the Arcules portal. Testing this section should take less than one hour. There are two different Settings dialogs to test: Basic Settings and Advanced Settings.

Basic Settings

This test confirms that all of the settings in the Basic Settings dialog are working as intended. This test should take less than fifteen minutes.

Test Procedure

  1. Open the Device Drawer.

  2. Press Settings -> Basic Settings.

  3. Change each setting available on the dialog.

  4. Press Save to apply settings and close the dialog.

  5. Reopen the dialog to check that the values saved are displayed when the dialog reopens.

Criteria for Success

  • The dialog successfully opens without any errors.

    • All fields should be populated.

  • The dialog should look similar to the image shown below:

  • When changing the Camera name, the new name should be displayed in the Device Drawer.

  • The Recording Resolution should display a list of resolutions supported by the device.

  • Higher frames per second (FPS) values may require a different Capture Mode (set in the AXIS UI).

  • When changing the recording settings, open the Recording page of the AXIS UI

    • There should be a current recording that began when the settings were changed.

      • This example shows 35 seconds.

    • Recording Resolution and Recording FPS should match the saved values.

Advanced Settings

This test confirms that all of the settings in the Advanced Settings dialog are working as intended. This test should take less than thirty minutes.

Test Procedure

  1. Open the Device Drawer.

  2. Press Settings -> Advanced Settings.

  3. Change each setting available on the dialog.

  4. Press Save to apply settings and close the dialog.

  5. Reopen the dialog to check that the values saved are displayed when the dialog reopens.

Criteria for Success

  • The dialog successfully opens without any errors.

  • The dialog should look similar to the images shown below:

    • If any settings are missing, please reference the official AXIS documentation to verify that the setting exists for the device under test.

  • Notes for specific settings:

    • Auto Rotation is not available on all models. Check the AXIS datasheet for details about the device under test. If supported, automatic rotation should be listed under Built-in installation aids and/or Image settings.

      • Example: AXIS P1467-LE supports automatic rotation but AXIS P3267-LE does not.

    • Autofocus causes the image to briefly lose focus before automatically adjusting back to a crisp image. Proper operation of this setting can be verified by looking at the image after it has completed.

    • The IR Cut Filter can cause infrared (IR) light to be emitted from the product. Please exercise caution.

  • Changing settings can cause recordings to restart or even cause gaps in recordings. This is expected.

Device Drawer Tabs

The purpose of this section is to test functionality for each of the tabs visible from the Device page in the Portal. This section assumes that the device has already been added to the Arcules portal. Testing this section should take about two hours. There are three different tabs to test: Overview, Storage, and Analytics.

Overview Tab

This test confirms that all of the features in the Overview tab are working as intended. Testing this section should take about an hour.

Test Procedure

  1. Open the Device Drawer.

    1. The Overview Tab should look similar to the image below:

  2. Use the video player to view Live video.

  3. Press the arrow icon on the video player to enter Playback (recorded video).

  4. Use the on-screen controls to verify Playback functionality.

    1. Navigate forward and backward to object detections.

    2. Jump forward/backward in time.

    3. Play/Pause.

    4. Forensic Search.

    5. Jump to specific time using the calendar.

    6. Advanced Features: Sharing, Export, Playback Speed, Timeline Display.

  5. Press Reveal Credentials in the HARDWARE DETAILS section.

  6. Copy the displayed password to the clipboard.

  7. Open the Status Page via the IP address or the Open Connection link in the CONNECTION section.

  8. Press the AXIS OS shortcut to open the AXIS UI.

  9. If prompted for credentials enter c2cadmin as the user and paste in the password copied earlier.

Criteria for Success

  • The video player works as expected for Live and Playback.

    • Correct behavior should be obvious; if anything feels wrong please report it.

  • Forensic Search results, if any, appear for the device under test.

  • Information in all panels is displayed correctly (e.g.: the MAC address is correct).

    • Note: Audio is not currently supported, so Not Available will always be displayed in this section.

  • The Status Page opens and functions as documented here.

  • Using the AXIS OS shortcut and entering the credentials should allow full access to the AXIS UI.

Storage Tab

This test confirms that all of the features in the Storage tab are working as intended. Testing this section should take about thirty minutes.

Test Procedure

  1. Open the Device Drawer.

  2. Select the Storage tab.

  3. Change the Video Retention setting in the Local Video Storage.

  4. Enable Cloud Upload.

  5. Leave the Video Retention for cloud storage set to 1 day.

  6. Press Apply Changes.

Criteria for Success

  • The Storage tab should appear similar to the image shown below:

  • The local retention time from the Arcules Portal should match the retention time in the AXIS UI under System -> Storage as shown below:

  • Verify Cloud Upload is working as intended by waiting about ten minutes, unplugging the camera, and checking that Playback is available for video uploaded to the cloud. The oldest video on the camera will be uploaded first.

Analytics Tab

This test confirms that all of the features in the Analytics tab are working as intended. Testing this section should take about thirty minutes.

Note: This section assumes that the device has already been added to the Arcules portal.

Test Procedure

  1. Open the Device Drawer

  2. Select the Analytics tab

  3. If desired, change the Resemblance, Regions, and Rules for both People and Vehicles

    1. The default settings should trigger on people and vehicles for the entire scene

Criteria for Success

  • The Analytics tab should display a recent thumbnail and look similar to the image below:

  • People and Vehicles should be accurately detected and visible in the Forensic Search as described in the Overview Tab section of this page.

  • If too many detections occur, changing the Resemblance and/or Region values should reduce the number of detections.

  • Creating Rules should work as expected.

Long-Running

The purpose of this section is to test that the device is operating normally after an extended period of time. Arcules recommends a duration of at least 14 days after being added to the Arcules Portal to ensure that the passwords on the device rotate at least once during the test period. After the test has concluded, verification should take less than thirty minutes. This section assumes that the device has already been added to the Arcules portal.

Test Procedure

  1. Open the Overview tab.

  2. Press Reveal Credentials.

  3. Make a note of the displayed password.

    1. The first few characters should suffice; do not write down the entire password.

  4. Open the Storage Tab.

  5. Customize the local and cloud retention settings in the Storage tab to desired values.

  6. Make a note of the amount of data recorded shown in Local Data Storage.

  7. Leave the camera powered on for the duration of the test.

    1. Log in to the portal as desired during the test to verify that the device is online.

    2. Make note of any network or power outages throughout the duration of the test.

  8. After the test concludes, open the Overview tab.

  9. Press Reveal Credentials.

  10. Make a note of the displayed password.

  11. Use the Playback feature of the to review video.

  12. Open the Storage tab.

    1. Verify that the Local Data Storage section accurately shows the amount of data recorded.

Criteria for Success

  • Before the Test:

    • The revealed password can be used to log in to the AXIS UI.

  • During the Test:

    • The device functions as expected.

  • After the Test:

    • The revealed password is different than the original if the test duration was at least 14 days.

    • The revealed password can be used to log in to the AXIS UI.

    • Playback shows video is available, including during network outages.

      • Video will not be available during any period where the camera was powered down.

    • Playback functions as expected (see Overview Tab test section for details).

    • The amount of data recorded during the test matches what is shown in Local Data Storage.

    • The amount of data in Local and Cloud storage does not exceed the retention time.

Checklist

This section contains a checklist for reference. Multiple related tests are condensed into single checklist items to keep the list short.

  1. Add & Remove Device.

  2. Basic & Advanced Settings.

  3. Live Video & Playback (Controls & Forensic Search).

  4. Overview Tab Information Correct (Hardware/Audio/Site).

  5. Access Status Page & AXIS UI.

  6. Local & Cloud Storage Settings.

  7. Local Data Storage Correct.

  8. Analytics Tab Thumbnail View.

  9. Analytics Person & Vehicle Detection.

  10. Long-Running: Playback & Retention Testing.

  11. Long-Running: Credentials Change and Work After 14+ Days.

Did this answer your question?