docs: Update browser profile user guide (#3034)

- Updates browser profiles user guide and separates into distinct pages
for overview, creating, and editing.
- Fixes workflow settings "Proxy" label to match UI.
- Adjusts mkdocs heading styling.

---------

Co-authored-by: Emma Segal-Grossman <[email protected]>
Co-authored-by: Tessa Walsh <[email protected]>

Author: 3 people

frontend/docs/docs/deploy/proxies.md

@@ -2,7 +2,7 @@
 
 Browsertrix can be configured to direct crawling traffic through dedicated proxy servers, allowing websites to be crawled from a specific geographic location regardless of where Browsertrix itself is deployed.
 
-The Browsertrix superadmin can configure which proxy servers are available to which organizations or if they are shared across all organizations, and users can [choose from one of the available proxies in each crawl workflow](../user-guide/workflow-setup.md#proxy). Users can also configure the default crawling proxy that will be used for the organization in organization-wide [Crawling Defaults](../user-guide/org-settings.md#crawling-defaults).
+The Browsertrix superadmin can configure which proxy servers are available to which organizations or if they are shared across all organizations, and users can [choose from one of the available proxies in each crawl workflow](../user-guide/workflow-setup.md#crawler-proxy-server). Users can also configure the default crawling proxy that will be used for the organization in organization-wide [Crawling Defaults](../user-guide/org-settings.md#crawling-defaults).
 
 This guide covers how to set up proxy servers for use with Browsertrix, as well as how to configure Browsertrix to make those proxies available to organizations.
 

frontend/docs/docs/overrides/.icons/bootstrap/lightbulb-fill.svg

@@ -134,12 +134,22 @@ h5 {
 }
 
 .md-typeset h1,
-h2,
-h3 {
-  font-weight: 650 !important;
+.md-typeset h2,
+.md-typeset h3,
+.md-typeset h4 {
+  font-weight: 600;
   font-variation-settings: "OPSZ" 35;
 }
 
+.md-typeset h3 {
+  font-size: 1.125em;
+}
+
+.md-typeset h4,
+.md-typeset h5 {
+  font-size: 0.875em;
+}
+
 .md-typeset {
   font-feature-settings:
     "ss04" off,

frontend/docs/docs/stylesheets/extra.css

@@ -0,0 +1,29 @@
+# Intro to Browser Profiles
+
+Browser profiles are saved instances of a web browsing session that can be used to configure a website before it is crawled.
+
+## Common Use Cases
+
+### Social Media Sign In
+
+Pre-configure a social media site to be logged in so that the crawler can access content that can only be viewed by logged-in users.
+
+!!! tip "Best Practices: Use an account created specifically for archiving a website"
+
+    We recommend creating dedicated accounts for archiving pages that require user registration but are otherwise public. This practice is sometimes referred to as a login wall. Login walls are commonly used by social media platforms.
+
+    Although dedicated accounts are not required to benefit from browser profiles, they can address the following potential issues:
+
+    - While usernames and passwords are never saved by Browsertrix, the private tokens that enable access to logged in content _are_ stored. Thus, anyone with access to your Browsertrix account, intentional or malicious, may be able to access the logged in content.
+
+    - Some websites may rate limit or lock accounts for reasons they deem to be suspicious, such as logging in from a new geographical location or if the site determines crawls to be robot activity.
+
+    - Personalized data such as cookies, location, etc. may be included in the resulting crawl.
+
+    - The logged in interface may display unwanted personally identifiable information such as a username or profile picture.
+
+    An exception to this practice is if your goal is to archive personalized or private content accessible only from designated accounts. In these instances we recommend changing the account's password after crawling is complete.
+
+### Hide Popup Prompts
+
+Websites may prompt users for a number of reasons before displaying the rest of the page, such as for age verification, informed consent requirements, or geographical location. Configure a browser profile to accept, dismiss, or otherwise hide these dialogs so that the content behind them is visible to the crawler.

frontend/docs/docs/user-guide/browser-profiles.md

@@ -0,0 +1,49 @@
+# Configure Sites
+
+Websites are configured through a temporary browser that is embedded directly in the Browsertrix interface. Every website that is visited using the embedded browser is added to the list of _Saved Sites_. When the embedded browser session ends, personalized data from the sites are collected into a profile. This profile of preconfigured sites can then be saved and used by multiple [crawl workflows](../crawl-workflows.md).
+
+The embedded browser is used during the process of [creating a new browser profile](./create-browser-profile.md) and when [editing an existing profile](./edit-browser-profile.md).
+
+## Use Cases
+
+### Website Sign In
+
+To crawl content as a logged in user, load the website you intend to archive in the embedded browser and sign in as you would on any other browser. Once the account has been logged in, confirm by accessing a page on the site that the crawler should have access to. You may need to periodically log in again as websites may log users out after a certain period of time.
+
+!!! tip "Tip: Crawl regularly to stay logged in"
+
+    Regularly running crawl workflows that use a browser profile can help to reduce the frequency with which logouts occur on some websites. Data such as cookies and sessions may be refreshed during crawling, and Browsertrix will automatically update the browser profile with this data when each crawl successfully finishes.
+
+### Hide Popups
+
+Load the website you intend to archive in the embedded browser and accept or otherwise dismiss the prompt. If the developers of the website have built the site in such a way that the result of your interaction is saved, the popup should remain hidden at crawl time. This can be confirmed by exiting the embedded browser session and then loading the site again.
+
+### Customize the Crawling Browser
+
+The embedded browser used to configure profiles is the same browser behind Browsertrix’s high-fidelity crawls. This enables advanced use cases like using a browser profile to customize the browser at crawl time. To view all available browser settings, load any site in the profile and then navigate to `brave://settings` in the embedded browser.
+
+!!! info "Advanced Use Case: Proceed with caution"
+    Customizing the crawler browser is for advanced use cases and it is not generally recommended to change these settings. We offer crawl-time browser customization like [ad blocking](../workflow-setup.md#block-ads-by-domain) and [language](../workflow-setup.md#language) in workflow settings. Changing browser settings directly in the profile may result in conflicting settings that are difficult to troubleshoot. If using this advanced feature, we recommend [adding clear metadata](./edit-browser-profile.md#edit-browser-profile-metadata) to the browser profile that describes the change.
+
+??? example "Example: Blocking page resources with Brave's Shields"
+    Whereas the crawler's scoping settings can be used to define which pages should be crawled, Brave's [Shields](https://brave.com/shields/) feature can block resources on pages from being loaded. By default, Shields will block [EasyList's cookie list](https://easylist.to/) but it can be set to block a number of other included lists under Brave `Settings > Shields > Filter Lists`.
+
+    _Custom Filters_ can also be useful for blocking sites with resources that aren't included in one of the known block lists.
+
+    The [uBlock Origin filter syntax](https://github.com/gorhill/uBlock/wiki/Static-filter-syntax) can be used for more specificity over what in-page resources should be blocked.
+
+    All of the browser's ad blocking and privacy features can be used in combination with the [_Block Ads by Domain_](../workflow-setup.md#block-ads-by-domain) crawler setting.
+
+## Saving the Profile
+
+After you are done interacting with the embedded browser, press _Save Profile_ (or _Create Profile_ for new browser profiles.)
+
+## Saved Sites
+
+All sites that are loaded in the embedded browser and then saved will appear in the _Saved Sites_ list. Select a site in the list to view or reconfigure the site in the embedded browser.
+
+## Load New URL
+
+You may want to load a URL that is not listed in the _Saved Sites_ to preview how a page may appear to the crawler, or to add a new site. Due to the nature of the embedded browser, it can be difficult to navigate between different websites if there are no hyperlinks between them. The easiest way to load a new URL is to press _Load New URL_ from the browser profile page and enter the URL.
+
+Although browser profiles have no limit on the number of saved sites, we recommend one site per browser profile to make troubleshooting crawls easier. An exception is when using a [URL List](../workflow-setup.md#list-of-pages) workflow to crawl multiple websites that require a profile, as we only allow one browser profile per workflow.

frontend/docs/docs/user-guide/browser-profiles/browser-profiles-overview.md

@@ -0,0 +1,29 @@
+# Create a New Browser Profile
+
+To create a new browser profile, press _New Browser Profile_ on the **Browser Profiles** page.
+
+## New Browser Profile Settings
+
+: ### Primary Site URL
+
+: The URL of the first page to visit in the embedded browser. For example, the login page for a social media website.
+
+: ### Profile Name
+
+: A custom name for the browser profile. The domain name of the _Primary Site URL_ will be used if this field is left blank.
+
+Depending on your organization settings, additional settings may be available:
+
+: ### Proxy Server
+
+: The proxy server to be used by the embedded browser as well as any crawl that uses this profile.
+
+    !!! tip "Implication for crawl workflows using proxies"
+    
+        When a browser profile is added to a crawl workflow, the browser profile’s proxy setting will take precedence over the crawl workflow’s [_Crawler Proxy Server_](../workflow-setup.md#crawler-proxy-server) setting. This prevents potential crawl failures that result from conflicting proxies.
+
+: ### Crawler Release Channel
+
+: For advanced use cases, you can specify a [Browsertrix Crawler](https://github.com/webrecorder/browsertrix-crawler) release that contains another version of the embedded browser, such as a beta version that may contain experimental features.
+
+Press _Start Browser_ to load the temporary embedded browser used to configure sites. It may take a few moments for the embedded browser to load. The browser profile will not be saved until _Create Profile_ is pressed.

frontend/docs/docs/user-guide/browser-profiles/configure-sites.md

@@ -0,0 +1,59 @@
+# Edit Browser Profile
+
+Sometimes websites will log users out by expiring cookies or login sessions after a period of time. Although running crawls on a regular basis can help keep websites logged in for longer, most websites will log users out after an extended period of time, be it a week or 30 days. In this case, the browser profile may not behave as expected at crawl time and will need to be reconfigured.
+
+To check or update the profile, go to the browser profile page and select _Load Profile_ from the action menu.
+
+!!! tip "Tip: Fail crawls early to identify logged-out profiles"
+
+    Enabling [_Fail Crawl If Not Logged In_](../workflow-setup.md#fail-crawl-if-not-logged-in) on a workflow can help identify which profiles need attention and prevent adding unwanted logged-out content to a collection.
+
+## Load / Edit Profile Settings
+
+: ### Primary Site
+
+: The primary site that is configured in the browser profile. A new primary site can be configured by choosing _New Site_ in the dropdown.
+
+: If the browser profile is in use by crawl workflows with a [crawl start URL](../workflow-setup.md#crawl-start-url-urls-to-crawl) that is not a saved site, a section titled _Suggestions from Related Workflows_ with suggested options will be displayed in the dropdown.
+
+: ### URL to Load
+
+: The URL of the first page in the primary site to visit in the embedded browser. For example, the login page for a social media website.
+
+: ### Reset previous configuration on save
+
+: If checked, all previously saved sites and their associated data will be removed from the browser profile. If your organization supports proxies, this will also enable choosing a different proxy server.
+
+Depending on your organization settings, additional settings may be available:
+
+: ### Proxy Server
+
+: The proxy server to be used by the embedded browser as well as any crawl that uses this profile.
+
+    ??? tip "Implication for crawl workflows using proxies"
+    
+        When a browser profile is added to a crawl workflow, the browser profile’s proxy setting will take precedence over the crawl workflow’s [_Crawler Proxy Server_](../workflow-setup.md#crawler-proxy-server) setting. This prevents potential crawl failures that result from conflicting proxies.
+
+: ### Crawler Release Channel
+
+: For advanced use cases, you can specify a [Browsertrix Crawler](https://github.com/webrecorder/browsertrix-crawler) release that contains another version of the embedded browser, such as a beta version that may contain experimental features.
+
+Press _Start Browser_ to load the temporary embedded browser. It may take a few moments for the embedded browser to load.
+
+When finished, press the _Save Profile_ button to return to the profile's details page. Profiles are automatically backed up on save if replica storage locations are configured.
+
+## Edit Browser Profile Metadata
+
+To edit the name, description, and tags, select _Edit Metadata_ from the action menu on the browser profile page.
+
+: ### Name
+
+: A custom name for the browser profile.
+
+: ### Description
+
+: A short description of the browser profile.
+
+: ### Tags
+
+: Tag the browser profile with additional metadata like category or keywords. Tags are displayed on the browser profiles list page.