app_dev/scan_api/scan_bulk_data_api

[giscus[bot]]

app_dev/scan_api/scan_bulk_data_api

https://docs.dev.sync.global/app_dev/scan_api/scan_bulk_data_api.html

[waynecollier-da]

Here's a sample of a cURL command that returns 1000 updates at 2025-02-20T13:31:28.405180Z

curl --location 'https://scan.sv-2.global.canton.network.digitalasset.com/api/scan/v1/updates' \
     --header "Content-Type: application/json" \
     --data '{
           "after": {
             "after_migration_id": 2,
             "after_record_time": "2025-02-20T13:31:28.405180Z"
           },
           "page_size": 1000,
           "daml_value_encoding": "compact_json"
         }' | jq
     

[waynecollier-da]

It's crude but a quick way to find updates at / near a given time. Adjust the record_time and the page_size to meet your needs.

[waynecollier-da]

Here's a bash script that returns the Canton Coin price for a range of record times.

Required inputs are

• A scan endpoint
• The migration ID for the record times
• The start and end record times

#!/usr/bin/env bash

# BSD Zero Clause License

# Copyright (c) 2025, Digital Asset
# All rights reserved.

# This software is provided "as is", without any warranty of any kind, either
# express or implied, including but not limited to the warranties of
# merchantability, fitness for a particular purpose and noninfringement.
# In no event shall the authors or copyright holders be liable for any claim,
# damages or other liability, whether in an action of contract, tort or otherwise,
# arising from, out of or in connection with the software or the use or other
# dealings in the software.

set -eou pipefail

if [ $# -ne 4 ]; then
   echo "Usage: $0 scan_url migration_id record_time_start record_time_end"
   exit 1
fi

scan_url="$1"
migration_id="$2"
record_time="$3"
record_time_end="$4"

SCAN_CACHE="$(mktemp)"

end=false

while [[ "$end" != "true" ]]; do
    curl -sSL --fail-with-body -H "content-type: application/json" "$scan_url/api/scan/v1/updates" -d "{\"after\": {\"after_migration_id\": $migration_id, \"after_record_time\": \"$record_time\"}, \"page_size\": 1000}" > "$SCAN_CACHE"
    for tx in $(jq -c '.transactions.[]' < "$SCAN_CACHE")
    do
        echo "$tx" | jq '.events_by_id.[] | select(.event_type == "created_event" and (.template_id | contains("OpenMiningRound"))) | {created_at, round: .create_arguments.round.number, amuletPrice: .create_arguments.amuletPrice}'
        migration_id="$(echo "$tx" | jq -r .migration_id)"
        record_time="$(echo "$tx" | jq -r .record_time)"
        if [[ "$record_time" > "$record_time_end" ]]; then
            end=true
            break
        fi
    done
done

[waynecollier-da]

Here's an example input, and sample output:

user@user-a7323 ~> ~/tmp/pricefeed.sh https://scan.sv-1.global.canton.network.sync.global 2 2025-03-21T00:00:00Z 2025-03-21T00:30:00Z
{
  "created_at": "2025-03-21T00:04:02.011991Z",
  "round": "37248",
  "amuletPrice": "0.0500000000"
}
{
  "created_at": "2025-03-21T00:14:09.326343Z",
  "round": "37249",
  "amuletPrice": "0.0500000000"
}
{
  "created_at": "2025-03-21T00:24:44.305283Z",
  "round": "37250",
  "amuletPrice": "0.0500000000"
}

[waynecollier-da]

Some useful Q&A about the Scan API:

Antoine Le Calvez
What's the maximum nesting depth of root/child event relations? In the transaction we went over it was just 1 (each root's children had no child), but can it be deeper?

kc
The depth of root/child event relation depends on the Daml model built in the template. So far my observation on the udpates are just 1. If you find a tx more than one, let me know and I can also take a look at it.

[waynecollier-da]

Useful Q&A:

Antoine Le Calvez
could there be transactions with the exact same timestamp?

Antoine Le Calvez
ah I figured it out:
For a given migration id, each update has a unique record time. The record time ranges of different migrations may overlap, i.e., it is not guaranteed that the maximum record time of one migration is smaller than the minimum record time of the next migration, and there may be two updates with the same record time but different migration ids.

[waynecollier-da]

Useful Q&A:

Antoine Le Calvez:
The after_record_time field is behaving oddly:

$ curl -s -x socks5://@localhost:5000 -H "Content-Type: application/json" -XPOST https://scan.sv-2.global.canton.network.digitalasset.com/api/scan/v1/updates -d '{"after": {"after_migration_id": 0, "after_record_time": "2025-04-29T00:00:00.000Z"}, "page_size": 1}' | jq -r ".transactions[0].record_time"
2024-10-16T13:24:18.493744Z

I request transactions with after_record_time set to today, but get back data from late October

kc:
For today you can set the after_migration_id to 2
when you set to 0 it will start with migration_id=1, which is last October.

Antoine Le Calvez:
what I want is to get all updates in chronological order, I had assumed I could keep after_migration_id at -1 and then just increase the record time, but that's doesn't seem to be possible. (edited)

kc:
my experience is that you can start with migration 0, and start with time some day say 2024-06024 (the first update), then you can have things from genesis.

{
    "page_size": 1000,
    "after": {
        "after_migration_id": 0,
        "after_record_time": "2024-06-01T00:00:00Z"
    }
}

something like that.
then you can check the latest record and tune the time for the next 1000 records.
when you hit a new migration (one in Oct, and one in Dec), you just change the after_migration_id to 1 and then 2 (latest). Then you should have the whole history.

Antoine Le Calvez:
I'm still confused by the semantics of after_migration_id and after_record_time.
I thought it was something like return all records with migration_id >= after_migration_id and record_time > after_record_time but it doesn't behave like that: I can get records with record_time < after_record_time.
Could you clarify those semantics for me?
For example,
after_migration_id = 0
after_record_time = 2024-10-16T14:00:00.000Z
and I get a record for 2024-10-16T13:24:18.493744Z

$ curl -s -x socks5://@localhost:5000 -H "Content-Type: application/json" -XPOST https://scan.sv-2.global.canton.network.digitalasset.com/api/scan/v1/updates -d '{"after": {"after_migration_id": 0, "after_record_time": "2024-10-16T14:00:00.000Z"}, "page_size": 1}' | jq -r ".transactions[] | [.record_time, .migration_id] | @csv"
"2024-10-16T13:24:18.493744Z",1

Bradley:
after_record_time only affects transactions with the same migration id as after_migration_id
Its a sub filter within that Migration Id.
So for the above request
after_migration_id = 0
after_record_time = 2024-10-16T14:00:00.000Z
Essentially what is happening is
It searches Migration ID 0 for a record time of 2024-10-16T14:00:00.000Z .
That returns an empty array
It then searches Migration ID 1 but does not include the sub filter of after_record_time and so returns the first result of Migration ID 1
I realize this is not entirely intuitive and have raised it with the Splice team.

Antoine Le Calvez:
thanks for the clarification!

[waynecollier-da]

Copying over some useful Q&A from a transient Slack discussion?

Antoine Le Calvez
Do all event ids follow the ${updateId}:${index} naming pattern? Is it something we can rely upon?

Bradley: Yes.