Merge pull request #179 from carbonplan/katamartin/offsets-db-additions

katamartin · web-flow · commit 389834ceed7c · 2025-04-23T13:51:10.000-04:00
Add OffsetsDB additions post
diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
@@ -24,7 +24,7 @@ jobs:
     steps:
       - uses: actions/checkout@v1
       - uses: actions/setup-python@v2.3.1
-      - uses: pre-commit/action@v2.0.3
+      - uses: pre-commit/action@v3.0.1
       - name: Set up Node.js 18.x
         uses: actions/setup-node@v2.5.1
         with:
diff --git a/components/mdx/page-components.js b/components/mdx/page-components.js
@@ -3,6 +3,18 @@ import dynamic from 'next/dynamic'
 // NOTE: This is a dynamically generated file based on the config specified under the
 //       `components` key in each post's frontmatter.
 const components = {
+  'offsets-db-additions': {
+    BeneficiarySearch: dynamic(() =>
+      import('../../posts/offsets-db-additions/beneficiary-search.js').then(
+        (mod) => mod.BeneficiarySearch || mod.default
+      )
+    ),
+    ProjectTypeSummary: dynamic(() =>
+      import('../../posts/offsets-db-additions/project-type-summary.js').then(
+        (mod) => mod.ProjectTypeSummary || mod.default
+      )
+    ),
+  },
   'counterfactual-accounting-update': {
     AccountingGraph: dynamic(() =>
       import(
diff --git a/package-lock.json b/package-lock.json
diff --git a/package.json b/package.json
@@ -54,6 +54,7 @@
     "react-dom": "^18.2.0",
     "rehype-slug": "^4.0.1",
     "remark-mdx-code-meta": "^2.0.0",
+    "swr": "^2.3.3",
     "theme-ui": "^0.15.3",
     "topojson-client": "^3.1.0",
     "zarr-js": "^1.0.0"
diff --git a/posts/offsets-db-additions.md b/posts/offsets-db-additions.md
@@ -0,0 +1,49 @@
+---
+version: 1.0.0
+title: We’ve added retirement user and project type data to OffsetsDB
+authors:
+  - Grayson Badgley
+  - Anderson Banihirwe
+  - Kata Martin
+date: 04-23-2025
+summary: We’ve expanded our database of offset data with searchable project types and details about who is using offset credits.
+components:
+  - name: BeneficiarySearch
+    src: ./beneficiary-search.js
+  - name: ProjectTypeSummary
+    src: ./project-type-summary.js
+---
+
+We’ve expanded the functionality of OffsetsDB, our database that organizes offset data from five of the largest offset registries. Starting today, you can access two new types of data: standardized information about the organizations and companies using offsets, and improved information about project types.
+
+Adding the retirement user data is a big change. Hardly a week goes by without someone emailing us asking how to figure out who benefits from retiring the offset credits sold by a particular project. It’s such a common question because it’s such an important question. A transparent offset market requires that we’re able to trace offset credits to companies and the specific claims (e.g., net zero) those companies make.
+
+Unfortunately, the global carbon market doesn’t always make that easy.
+
+Part of the problem is that there’s no requirement to disclose who’s using which offsets. Anyone who uses a voluntary offset to make an environmental claim has the _option_ to publicly disclose that credit retirement. But most don’t. Of 1.34 billion credit retirements tracked in OffsetsDB, only 0.67 billion include information about who benefits from that retirement. In other words, we know next to nothing about who used roughly half the offset credits within the voluntary carbon market.
+
+And the half we do have data for is a bit of a mess. Rather than existing in a single place, every registry has its own version of user data, with no consistency between how different registries keep their records. In fact, even individual registries don’t organize their own retirement user data in a consistent way.
+
+Say you want to know how many credits Delta, the commercial airline, has retired. First, you have to gather retirement data from all the registries — there’s no guarantee Delta will have only purchased from one. Then you have to run multiple searches of those registries’ data, using a variety of different possible ways of writing the airline’s name. A retirement could be linked to “Delta,” or “Delta Airlines,” or “Delta Air Lines.” In fact, Delta often retires credits under “DL,” which is its airline code used for its flight numbers. Lack of standard schema makes it painstaking to answer even simple questions about trends in offset usage. It also doesn’t help that we’re constantly getting new retirement data with no guarantees about what format _that_ data will use.
+
+Suffice it to say, none of this is good for market transparency.
+
+<Figure>
+  <BeneficiarySearch />
+</Figure>
+
+Our updates to OffsetsDB bring both increased standardization and searchability to user data. For this initial release, we used a mostly manual process to pull information from offset registries and fit it to a common format. While we’ve automated parts of the process, we foresee some amount of manual intervention going forward. In fact, this first release only standardizes a little over two-thirds of the 0.67 billion classifiable credits in the database.
+
+It’s a big lift, but we think it’s worth the ongoing effort. Claims backed by offsets can’t be taken seriously if there’s no way to tie them to specific credits that come from specific projects. With that in mind, we’re excited to see standards bodies within the carbon market, such as ICVCM and SBTi, start to advocate for full, complete, and standardized disclosures of retirement user data. Hopefully, over the long term, this will make user data less messy — and make our job easier.
+
+Our other change to OffsetsDB is smaller in scope, but still important. Previously, you could search OffsetsDB by project categories — broad groupings of projects, like Forests or Renewable Energy. We’ve now added project types that further refine those categories, allowing more precise searches. So rather than looking at everything categorized as Forest, you can now distinguish reforestation projects from improved forest management projects.
+
+<Figure>
+  <ProjectTypeSummary />
+</Figure>
+
+While the distinction between categories and types might seem minor, it takes a fair amount of work to assign projects to their appropriate type. OffsetsDB originally sorted projects into categories based on their protocol. But a protocol such as ACM0002 — Grid-connected Electricity Generation from Renewable Sources — could include electricity generated by wind, solar, or hydroelectric facilities. To sort these projects by type, we have to turn to project specific paperwork. And there is a lot of paperwork to go through — OffsetsDB tracks more than 10,000 projects.
+
+Thankfully, we didn’t have to start from scratch. That’s because the folks over at the [Berkeley Carbon Trading Project](https://gspp.berkeley.edu/research-and-impact/centers/cepp/projects/berkeley-carbon-trading-project) maintain a similar offsets database that already contains detailed information about project types. And, what’s more, they publish their project-type information using a [permissive license](https://creativecommons.org/licenses/by/4.0/), which means we can freely incorporate their work into OffsetsDB. That’s wonderful because project-type data is the sort of thing you only need to create once. This underscores the value we see in open science — once someone classifies a project and shares the data, no one else should ever have to do that work again.
+
+As with previous installments of OffsetsDB, you’re free to access the data both through our the browser-based [database tool](https://carbonplan.org/research/offsets-db) or by [downloading the data](https://offsets-db-data.readthedocs.io/en/latest/data-access.html) and playing around with it yourself. Keep track of our progress on [GitHub](https://github.com/carbonplan/offsets-db-data) or reach out at [hello@carbonplan.org](mailto:hello@carbonplan.org) if you have any questions.
diff --git a/posts/offsets-db-additions/beneficiary-search.js b/posts/offsets-db-additions/beneficiary-search.js
@@ -0,0 +1,214 @@
+import { Box, Flex, IconButton } from 'theme-ui'
+import { Row, Column, Input, Button } from '@carbonplan/components'
+import { RotatingArrow, Search, X } from '@carbonplan/icons'
+import { useEffect, useState } from 'react'
+import { format } from 'd3-format'
+import useSWR from 'swr'
+
+export const formatValue = (value) => {
+  if (value < 1000) {
+    return value
+  } else {
+    let result = format('.3s')(value)
+    if (value >= 1e9) {
+      return result.replace('G', 'B')
+    }
+    return result.toUpperCase(0)
+  }
+}
+
+const sx = {
+  row: {
+    borderStyle: 'solid',
+    borderWidth: '0px',
+    borderTopWidth: '1px',
+    borderColor: 'muted',
+    height: '100%',
+    py: [3, 3, 3, 4],
+  },
+  label: {
+    fontFamily: 'heading',
+    letterSpacing: 'smallcaps',
+    textTransform: 'uppercase',
+    fontSize: [2, 2, 2, 3],
+  },
+  value: {
+    fontFamily: 'faux',
+    letterSpacing: 'faux',
+    textTransform: 'uppercase',
+    color: 'purple',
+    fontSize: [3, 3, 3, 4],
+  },
+}
+
+async function getTransactions([, search]) {
+  try {
+    const reqUrl = new URL(
+      `https://carbonplan.org/research/offsets-db/api/query`
+    )
+    const params = new URLSearchParams()
+    params.append('path', 'credits')
+    params.append('per_page', 1)
+    params.append('transaction_type', 'retirement')
+    if (search) {
+      params.append('beneficiary_search', search.trim())
+      params.append(
+        'beneficiary_search_fields',
+        'retirement_beneficiary_harmonized'
+      )
+      params.append('beneficiary_search_fields', 'retirement_account')
+      params.append('beneficiary_search_fields', 'retirement_beneficiary')
+      params.append('beneficiary_search_fields', 'retirement_note')
+      params.append('beneficiary_search_fields', 'retirement_reason')
+    }
+    reqUrl.search = params.toString()
+    const serverRes = await fetch(reqUrl, {
+      credentials: 'include',
+    })
+    if (!serverRes.ok) {
+      throw new Error(
+        `API request failed: ${serverRes.status} ${serverRes.statusText}`
+      )
+    }
+    const result = await serverRes.json()
+    return result.pagination.total_entries
+  } catch (e) {
+    throw new Error('Unexpected error')
+  }
+}
+
+export const useDebounce = (value, delay = 100) => {
+  const [debounced, setDebounced] = useState(value)
+
+  useEffect(() => {
+    const timeoutID = setTimeout(() => {
+      setDebounced(value)
+    }, delay)
+
+    return () => {
+      clearTimeout(timeoutID)
+    }
+  }, [value, delay])
+
+  return debounced
+}
+
+const BeneficiarySearch = () => {
+  const [search, setSearch] = useState('Delta Airlines')
+  const { data: transactions, isLoading } = useSWR(
+    ['/credits', useDebounce(search)],
+    getTransactions,
+    {
+      revalidateOnFocus: false,
+      revalidateIfStale: false,
+    }
+  )
+  return (
+    <>
+      <Row columns={6} sx={sx.row}>
+        <Column start={1} width={[3, 2, 2, 2]}>
+          <Box sx={sx.label}>
+            <Flex
+              as='label'
+              htmlFor='beneficiary_search'
+              sx={{ gap: 2, alignItems: 'center' }}
+            >
+              User <Search sx={{ width: 14 }} />
+            </Flex>
+          </Box>
+        </Column>
+        <Column start={[4, 3, 3, 3]} width={[3, 4, 4, 4]}>
+          <Flex sx={{ justifyContent: 'space-between' }}>
+            <Input
+              id='beneficiary_search'
+              placeholder='enter a search term'
+              color='purple'
+              value={search}
+              onChange={(e) => setSearch(e.target.value)}
+              sx={{
+                ...(search ? sx.value : { fontFamily: 'mono', fontSize: 2 }),
+                height: '22px',
+                textTransform: 'none',
+                borderBottom: 0,
+                borderColor: 'muted',
+                flexGrow: 1,
+              }}
+            />
+            <IconButton
+              sx={{
+                cursor: 'pointer',
+                p: [0],
+                mt: ['3px', '3px', '3px', '4px'],
+                mx: [2],
+                width: 18,
+                height: 18,
+                flexShrink: 0,
+              }}
+              onKeyDown={(e) => {
+                if (e.key === 'Enter') {
+                  setSearch('')
+                }
+              }}
+              onClick={() => {
+                setSearch('')
+              }}
+            >
+              <X
+                sx={{
+                  strokeWidth: 1.5,
+                  color: 'secondary',
+                  transition: 'stroke 0.15s',
+                  '@media (hover: hover) and (pointer: fine)': {
+                    '&:hover': {
+                      stroke: 'primary',
+                    },
+                  },
+                }}
+              />
+            </IconButton>
+          </Flex>
+        </Column>
+      </Row>
+      <Row columns={6} sx={sx.row}>
+        <Column start={1} width={[3, 2, 2, 2]}>
+          <Box sx={sx.label}>Retirements</Box>
+        </Column>
+        <Column start={[4, 3, 3, 3]} width={[3, 4, 4, 4]}>
+          <Flex
+            sx={{
+              height: '100%',
+              alignItems: 'baseline',
+              gap: 2,
+              mt: [0, 0, 0, '-6px'],
+            }}
+          >
+            <Box sx={sx.value}>
+              {!isLoading && typeof transactions === 'number'
+                ? formatValue(transactions)
+                : '—'}
+            </Box>
+            <Box sx={{ fontFamily: 'mono' }}>
+              {transactions === 1 ? 'transaction' : 'transactions'}
+            </Box>
+          </Flex>
+        </Column>
+      </Row>
+      <Row columns={6} sx={{ ...sx.row, pb: 0 }}>
+        <Column start={[4, 3, 3, 3]} width={[3, 4, 4, 4]}>
+          <Button
+            suffix={<RotatingArrow />}
+            size='xs'
+            inverted
+            href={`https://carbonplan.org/research/offsets-db/transactions${
+              search ? `?beneficiary=${search}` : ''
+            }`}
+          >
+            View in OffsetsDB
+          </Button>
+        </Column>
+      </Row>
+    </>
+  )
+}
+
+export default BeneficiarySearch
diff --git a/posts/offsets-db-additions/project-type-summary.js b/posts/offsets-db-additions/project-type-summary.js
