|
1 | 1 | OSG Networking Area |
2 | 2 | =================== |
3 | 3 |
|
4 | | -*Welcome to OSG Networking !* This is an entry point for those interested in Networking |
5 | | -in OSG/WLCG or for those OSG/WLCG users experiencing network problems. It provides an |
6 | | -overview of the networking goals, plans and various activities and subtopics underway |
7 | | -regarding networking in the *Open Science Grid (OSG)* and *World-wide LHC Computing Grid (WLCG)*, |
8 | | -operated as a joint project. This area started in June 2012 with initial focus on the network |
9 | | -monitoring as monitoring is critical to provide needed visibility into existing networks and site |
10 | | -connectivity. OSG is working to provide needed networking information and tools for users, sites |
11 | | -and experiments/VOs. |
12 | | - |
13 | | -This documentation is divided into several sub-sections, each covering a specific area of activities. |
14 | | - |
15 | | -Network Monitoring in WLCG and OSG (perfSONAR) |
16 | | ------------------------------------------------ |
17 | | - |
18 | | -WLCG and OSG jointly operate a network of `perfSONAR` agents deployed world-wide, which provides an |
19 | | -open platform that can be used to baseline network performance and debug any potential issues. The |
20 | | -following subsections provide details on the motivation, deployment and operations of the perfSONARs |
21 | | -in WLCG/OSG: |
22 | | - |
23 | | -- [Motivation](perfsonar-in-osg.md) - overview, core concepts, motivation |
24 | | -- [Deployment Guide](perfsonar/deployment-models.md) - deployment models and options, hardware requirements |
25 | | -- [Installation and Administration Guide](perfsonar/installation.md) - installation, configuration and maintanance |
26 | | -- [Frequently Asked Questions](perfsonar/faq.md) |
27 | | - |
28 | | -Network Troubleshooting |
29 | | ------------------------ |
30 | | - |
31 | | -Users with network issues should check the [troubleshooting link](network-troubleshooting.md) below |
32 | | -for initial guidance on how best to get their issue resolved. In addition, you can refer to the |
33 | | -[ESNet network performance guide](https://fasterdata.es.net/performance-testing/troubleshooting/network-troubleshooting-quick-reference-guide/) |
34 | | -for a detailed instructions on how to identify and isolate network performance issues using perfSONAR. |
35 | | - |
36 | | -Host and Network Tuning |
37 | | ------------------------ |
38 | | - |
39 | | -- [Fasterdata-aligned host/network tuning (EL9)](host-network-tuning.md) — summarizes ESnet guidance and includes an audit/apply script. |
40 | | - |
41 | | -Network Services |
42 | | ----------------- |
43 | | - |
44 | | -OSG operates an advanced platform to collect, store, publish and analyse the network monitoring data it gathers from perfSONAR and other locations. All measurements are collected and available via streaming or through APIs. The following services are available: |
45 | | - |
46 | | -- [perfSONAR infrastructure monitoring](perfsonar/psetf.md) - monitors state of perfSONAR network and reports on availability of core services |
47 | | -- [*OSG Distributed Network Datastore*](https://atlas-kibana.mwt2.org/s/networking/app/kibana#/dashboards?notFound=dashboard&_g=()) - distributed datastore based on ElasticSearch holding all the network measurements and providing an API to expose them via JSON is available at two locations (University of Chicago and University of Nebraska). |
48 | | -- *OSG pSConfig Web Admin (PWA)* - centralized configuration of the tests performed by the OSG/WLCG perfSONAR infrastructure . In case you'd like to start/manage particular mesh, please contact our support channels to get access. |
49 | | -- *OSG Dashboards* [http://maddash.aglt2.org](https://maddash.aglt2.org) - set of dashboards showing an overview of the network state as seen by the perfSONAR infrastructure (NOTE: this instance is being deprecated and we plan to introduce dashboards that will replace MaDDash over the coming 2023-2024 year). |
50 | | -- [*WLCG Dashboards*](https://monit-grafana-open.cern.ch/d/MwuxgogIk/wlcg-site-network?var-bin=1h&orgId=16)) - set of dashboards showing WLCG and OSG network performance by combining multiple sources of data including perfSONAR, FTS, ESNet/LHCOPN traffic, etc. |
51 | | - |
52 | | -Network Analytics |
53 | | ------------------ |
54 | | - |
55 | | -University of Chicago has set up an [**analytics platform**](<https://twiki.cern.ch/twiki/bin/view/AtlasComputing/ATLASAnalytics>) using `ElasticSearch` and `Kibana4` as well as `Jupyter` that can be used to access and analyse all the existing network measurements. |
56 | | - |
57 | | -Support and Feedback |
58 | | --------------------- |
59 | | - |
60 | | -If you suspect a network problem and wish to follow up on it, we have a number of tools |
61 | | -available. We have a [ToolkitInfo](https://toolkitinfo.opensciencegrid.org/) page that can |
62 | | -help you find resources to identify and explore problems. In general, networks problems are |
63 | | -best resolved by opening a ticket with your site's network provider (see |
64 | | -<https://osg-htc.org/networking/network-troubleshooting/>). If you want WLCG/OSG specific |
65 | | -support, please open a ticket with the appropriate support unit: For `OSG` sites please open |
66 | | -a ticket with [GOC](https://support.opensciencegrid.org/support/home); For `WLCG` sites |
67 | | -please open a [GGUS](https://ggus.eu/) ticket to `WLCG Network Throughput` support unit. If |
68 | | -you'd like to get help in setting up a WLCG/OSG perfSONAR instance please open a ticket with |
69 | | -[GOC](https://support.opensciencegrid.org/support/home) or via [GGUS](https://ggus.eu/) to |
70 | | -WLCG perfSONAR support. If you have problems or questions specific to perfSONAR, please email |
71 | | -the perfSONAR user [mailing list](https://lists.internet2.edu/sympa/info/perfsonar-user). For |
72 | | -any other requests or to provide feedback, please open a ticket at [GGUS](https://ggus.eu/) |
73 | | -and mention OSG networking. |
74 | | - |
75 | | -References |
76 | | ----------- |
77 | | - |
78 | | -- ESNet network performance tuning and debugging <https://fasterdata.es.net/> |
79 | | -- [perfSONAR](http://docs.perfsonar.net/) toolkit is part of the [perfSONAR](https://www.perfsonar.net/) project. |
80 | | -- **OSG/WLCG mesh configuration interface** is available at <https://psconfig.opensciencegrid.org> |
81 | | -- **OSG dashboard instance** <https://maddash.aglt2.org> (NOTE: deprecated replacement) |
82 | | -- **OSG perfSONAR infrastructure monitoring** <https://psetf.aglt2.org/etf/check_mk/> |
83 | | -- **OSG Analytics platform** <https://atlas-kibana.mwt2.org/s/networking/app/kibana> |
84 | | -- **WLCG dashboards** <https://monit-grafana-open.cern.ch/d/MwuxgogIk/wlcg-site-network?var-bin=1h&orgId=16> |
| 4 | +*Welcome to OSG Networking!* This is your entry point for networking in the *Open Science Grid (OSG)* and *World-wide LHC Computing Grid (WLCG)*. Whether you're deploying perfSONAR, troubleshooting network issues, or exploring our monitoring infrastructure, we've organized the documentation around common user journeys. |
| 5 | + |
| 6 | +## Get Started |
| 7 | + |
| 8 | +Choose the path that matches your goal: |
| 9 | + |
| 10 | +<div class="grid cards" markdown> |
| 11 | + |
| 12 | +- :rocket: **Deploy perfSONAR** |
| 13 | + |
| 14 | + --- |
| 15 | + |
| 16 | + Quick, tested steps to deploy a perfSONAR testpoint for OSG/WLCG monitoring. |
| 17 | + |
| 18 | + **Time:** 30-60 minutes |
| 19 | + **Skill level:** Systems administrator |
| 20 | + |
| 21 | + [:octicons-arrow-right-24: Quick Deploy Guide](personas/quick-deploy/landing.md) |
| 22 | + |
| 23 | +- :wrench: **Troubleshoot Network Issues** |
| 24 | + |
| 25 | + --- |
| 26 | + |
| 27 | + Triage checklist and playbooks for diagnosing network problems. |
| 28 | + |
| 29 | + **Time:** Variable |
| 30 | + **Skill level:** Network operator/admin |
| 31 | + |
| 32 | + [:octicons-arrow-right-24: Troubleshooting Guide](personas/troubleshoot/landing.md) |
| 33 | + |
| 34 | +- :telescope: **Understand the System** |
| 35 | + |
| 36 | + --- |
| 37 | + |
| 38 | + Architecture, data pipelines, and research documentation. |
| 39 | + |
| 40 | + **Time:** Reading/reference |
| 41 | + **Skill level:** Developer/researcher |
| 42 | + |
| 43 | + [:octicons-arrow-right-24: Architecture & Research](personas/research/landing.md) |
| 44 | + |
| 45 | +</div> |
| 46 | + |
| 47 | +## About OSG/WLCG Network Monitoring |
| 48 | + |
| 49 | +WLCG and OSG jointly operate a worldwide network of `perfSONAR` agents that provide an open platform for baselining network performance and debugging issues. This monitoring infrastructure is critical for providing visibility into networks and site connectivity. |
| 50 | + |
| 51 | +**Key capabilities:** |
| 52 | + |
| 53 | +- Automated bandwidth and latency testing between sites |
| 54 | +- Centralized measurement storage and analytics |
| 55 | +- Integration with WLCG/OSG dashboards and alerting |
| 56 | +- Community-maintained test meshes |
| 57 | + |
| 58 | +[Learn more about perfSONAR in OSG/WLCG](perfsonar-in-osg.md){ .md-button } |
| 59 | + |
| 60 | +## Network Services & Data |
| 61 | + |
| 62 | +OSG operates an advanced platform to collect, store, publish and analyze network monitoring data from perfSONAR and other sources. All measurements are available via streaming APIs and dashboards: |
| 63 | + |
| 64 | +- **[perfSONAR Infrastructure Monitoring](perfsonar/psetf.md)** - monitors perfSONAR network health and service availability |
| 65 | +- **[OSG Network Datastore](osg-network-services.md)** - distributed ElasticSearch datastore with JSON API (University of Chicago and University of Nebraska) |
| 66 | +- **OSG pSConfig Web Admin** - centralized test mesh configuration (contact support for access) |
| 67 | +- **[WLCG Dashboards](https://monit-grafana-open.cern.ch/d/MwuxgogIk/wlcg-site-network?var-bin=1h&orgId=16)** - comprehensive performance dashboards combining perfSONAR, FTS, and network traffic data |
| 68 | +- **[Analytics Platform](osg-network-analytics.md)** - ElasticSearch/Kibana/Jupyter for analyzing measurements |
| 69 | + |
| 70 | +!!! note "MaDDash Deprecation" |
| 71 | + The legacy MaDDash instance at maddash.aglt2.org is deprecated. Use WLCG Grafana dashboards instead. |
| 72 | + |
| 73 | +## Support and Feedback |
| 74 | + |
| 75 | +**For network problems:** |
| 76 | + |
| 77 | +1. Start with the [Troubleshooting Guide](network-troubleshooting.md) or [ToolkitInfo](https://toolkitinfo.opensciencegrid.org/) |
| 78 | +2. Contact your site's network provider |
| 79 | +3. For OSG-specific support: [GOC ticket](https://support.opensciencegrid.org/support/home) |
| 80 | +4. For WLCG-specific support: [GGUS ticket](https://ggus.eu/) to "WLCG Network Throughput" or "WLCG perfSONAR support" |
| 81 | + |
| 82 | +**For perfSONAR questions:** [perfSONAR user mailing list](https://lists.internet2.edu/sympa/info/perfsonar-user) |
| 83 | + |
| 84 | +## Quick Links |
| 85 | + |
| 86 | +- [perfSONAR Documentation](https://docs.perfsonar.net/) | [perfSONAR Project](https://www.perfsonar.net/) |
| 87 | +- [ESNet Fasterdata Guide](https://fasterdata.es.net/) |
| 88 | +- [OSG/WLCG Mesh Configuration](https://psconfig.opensciencegrid.org) |
| 89 | +- [perfSONAR Infrastructure Monitoring](https://psetf.aglt2.org/etf/check_mk/) |
| 90 | +- [OSG Analytics Platform](https://atlas-kibana.mwt2.org/s/networking/app/kibana) |
| 91 | +- [WLCG Grafana Dashboards](https://monit-grafana-open.cern.ch/d/MwuxgogIk/wlcg-site-network?var-bin=1h&orgId=16) |
0 commit comments