Skip to content

Commit 4f99c2a

Browse files
committed
MINIFICPP-2587 Add Sparkplug support for ConsumeMQTT processor
- Extract protobuf library to be built separately - Upgrade grpc library to v1.72.2 - Change RecordReader interface to expect InputStream instead of FlowFile and ProcessSession - Add SparkplugReader record reader controller - Add Record Reader and Record Writer properties to ConsumeMQTT processor - Add "Add Attributes As Fields" to add flow file attributes as record fields if records are used in ConsumeMQTT - Make ConsumeMQTT trigger unit testable - Add new attributes to ConsumeMQTT created flow files to match NiFi - Implement record reader and writer usage in ConsumeMQTT
1 parent 37a1f4f commit 4f99c2a

38 files changed

+1548
-618
lines changed

CONTROLLERS.md

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -29,6 +29,7 @@ limitations under the License.
2929
- [PersistentMapStateStorage](#PersistentMapStateStorage)
3030
- [RocksDbStateStorage](#RocksDbStateStorage)
3131
- [SmbConnectionControllerService](#SmbConnectionControllerService)
32+
- [SparkplugBReader](#SparkplugBReader)
3233
- [SSLContextService](#SSLContextService)
3334
- [UpdatePolicyControllerService](#UpdatePolicyControllerService)
3435
- [VolatileMapStateStorage](#VolatileMapStateStorage)
@@ -277,6 +278,20 @@ In the list below, the names of required properties appear in bold. Any other pr
277278
| Password | | | The password used for authentication. Required if Username is set.<br/>**Sensitive Property: true** |
278279

279280

281+
## SparkplugBReader
282+
283+
### Description
284+
285+
Reads Sparkplug B messages and turns them into individual Record objects. The reader expects a single Sparkplug B payload in a read operation, which is a protobuf-encoded binary message. This reader is typically used with MQTT processors like ConsumeMQTT.
286+
287+
### Properties
288+
289+
In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the NiFi Expression Language.
290+
291+
| Name | Default Value | Allowable Values | Description |
292+
|------|---------------|------------------|-------------|
293+
294+
280295
## SSLContextService
281296

282297
### Description

LICENSE

Lines changed: 281 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3231,7 +3231,7 @@ The above copyright notice and this permission notice shall be included in all c
32313231
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
32323232

32333233

3234-
This product bundles 'protobuf' within 'gRPC' under a 3-Clause BSD license:
3234+
This product bundles 'protobuf' under a 3-Clause BSD license:
32353235

32363236
Copyright 2008 Google Inc. All rights reserved.
32373237

@@ -3485,3 +3485,283 @@ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
34853485
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
34863486
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
34873487
SOFTWARE.
3488+
3489+
This product bundles 'Sparkplug B' protobuf file from the 'Eclipse Tahu' project which is available under the Eclipse Public License - v 2.0 license below.
3490+
3491+
Eclipse Public License - v 2.0
3492+
3493+
THE ACCOMPANYING PROGRAM IS PROVIDED UNDER THE TERMS OF THIS ECLIPSE
3494+
PUBLIC LICENSE ("AGREEMENT"). ANY USE, REPRODUCTION OR DISTRIBUTION
3495+
OF THE PROGRAM CONSTITUTES RECIPIENT'S ACCEPTANCE OF THIS AGREEMENT.
3496+
3497+
1. DEFINITIONS
3498+
3499+
"Contribution" means:
3500+
3501+
a) in the case of the initial Contributor, the initial content
3502+
Distributed under this Agreement, and
3503+
3504+
b) in the case of each subsequent Contributor:
3505+
i) changes to the Program, and
3506+
ii) additions to the Program;
3507+
where such changes and/or additions to the Program originate from
3508+
and are Distributed by that particular Contributor. A Contribution
3509+
"originates" from a Contributor if it was added to the Program by
3510+
such Contributor itself or anyone acting on such Contributor's behalf.
3511+
Contributions do not include changes or additions to the Program that
3512+
are not Modified Works.
3513+
3514+
"Contributor" means any person or entity that Distributes the Program.
3515+
3516+
"Licensed Patents" mean patent claims licensable by a Contributor which
3517+
are necessarily infringed by the use or sale of its Contribution alone
3518+
or when combined with the Program.
3519+
3520+
"Program" means the Contributions Distributed in accordance with this
3521+
Agreement.
3522+
3523+
"Recipient" means anyone who receives the Program under this Agreement
3524+
or any Secondary License (as applicable), including Contributors.
3525+
3526+
"Derivative Works" shall mean any work, whether in Source Code or other
3527+
form, that is based on (or derived from) the Program and for which the
3528+
editorial revisions, annotations, elaborations, or other modifications
3529+
represent, as a whole, an original work of authorship.
3530+
3531+
"Modified Works" shall mean any work in Source Code or other form that
3532+
results from an addition to, deletion from, or modification of the
3533+
contents of the Program, including, for purposes of clarity any new file
3534+
in Source Code form that contains any contents of the Program. Modified
3535+
Works shall not include works that contain only declarations,
3536+
interfaces, types, classes, structures, or files of the Program solely
3537+
in each case in order to link to, bind by name, or subclass the Program
3538+
or Modified Works thereof.
3539+
3540+
"Distribute" means the acts of a) distributing or b) making available
3541+
in any manner that enables the transfer of a copy.
3542+
3543+
"Source Code" means the form of a Program preferred for making
3544+
modifications, including but not limited to software source code,
3545+
documentation source, and configuration files.
3546+
3547+
"Secondary License" means either the GNU General Public License,
3548+
Version 2.0, or any later versions of that license, including any
3549+
exceptions or additional permissions as identified by the initial
3550+
Contributor.
3551+
3552+
2. GRANT OF RIGHTS
3553+
3554+
a) Subject to the terms of this Agreement, each Contributor hereby
3555+
grants Recipient a non-exclusive, worldwide, royalty-free copyright
3556+
license to reproduce, prepare Derivative Works of, publicly display,
3557+
publicly perform, Distribute and sublicense the Contribution of such
3558+
Contributor, if any, and such Derivative Works.
3559+
3560+
b) Subject to the terms of this Agreement, each Contributor hereby
3561+
grants Recipient a non-exclusive, worldwide, royalty-free patent
3562+
license under Licensed Patents to make, use, sell, offer to sell,
3563+
import and otherwise transfer the Contribution of such Contributor,
3564+
if any, in Source Code or other form. This patent license shall
3565+
apply to the combination of the Contribution and the Program if, at
3566+
the time the Contribution is added by the Contributor, such addition
3567+
of the Contribution causes such combination to be covered by the
3568+
Licensed Patents. The patent license shall not apply to any other
3569+
combinations which include the Contribution. No hardware per se is
3570+
licensed hereunder.
3571+
3572+
c) Recipient understands that although each Contributor grants the
3573+
licenses to its Contributions set forth herein, no assurances are
3574+
provided by any Contributor that the Program does not infringe the
3575+
patent or other intellectual property rights of any other entity.
3576+
Each Contributor disclaims any liability to Recipient for claims
3577+
brought by any other entity based on infringement of intellectual
3578+
property rights or otherwise. As a condition to exercising the
3579+
rights and licenses granted hereunder, each Recipient hereby
3580+
assumes sole responsibility to secure any other intellectual
3581+
property rights needed, if any. For example, if a third party
3582+
patent license is required to allow Recipient to Distribute the
3583+
Program, it is Recipient's responsibility to acquire that license
3584+
before distributing the Program.
3585+
3586+
d) Each Contributor represents that to its knowledge it has
3587+
sufficient copyright rights in its Contribution, if any, to grant
3588+
the copyright license set forth in this Agreement.
3589+
3590+
e) Notwithstanding the terms of any Secondary License, no
3591+
Contributor makes additional grants to any Recipient (other than
3592+
those set forth in this Agreement) as a result of such Recipient's
3593+
receipt of the Program under the terms of a Secondary License
3594+
(if permitted under the terms of Section 3).
3595+
3596+
3. REQUIREMENTS
3597+
3598+
3.1 If a Contributor Distributes the Program in any form, then:
3599+
3600+
a) the Program must also be made available as Source Code, in
3601+
accordance with section 3.2, and the Contributor must accompany
3602+
the Program with a statement that the Source Code for the Program
3603+
is available under this Agreement, and informs Recipients how to
3604+
obtain it in a reasonable manner on or through a medium customarily
3605+
used for software exchange; and
3606+
3607+
b) the Contributor may Distribute the Program under a license
3608+
different than this Agreement, provided that such license:
3609+
i) effectively disclaims on behalf of all other Contributors all
3610+
warranties and conditions, express and implied, including
3611+
warranties or conditions of title and non-infringement, and
3612+
implied warranties or conditions of merchantability and fitness
3613+
for a particular purpose;
3614+
3615+
ii) effectively excludes on behalf of all other Contributors all
3616+
liability for damages, including direct, indirect, special,
3617+
incidental and consequential damages, such as lost profits;
3618+
3619+
iii) does not attempt to limit or alter the recipients' rights
3620+
in the Source Code under section 3.2; and
3621+
3622+
iv) requires any subsequent distribution of the Program by any
3623+
party to be under a license that satisfies the requirements
3624+
of this section 3.
3625+
3626+
3.2 When the Program is Distributed as Source Code:
3627+
3628+
a) it must be made available under this Agreement, or if the
3629+
Program (i) is combined with other material in a separate file or
3630+
files made available under a Secondary License, and (ii) the initial
3631+
Contributor attached to the Source Code the notice described in
3632+
Exhibit A of this Agreement, then the Program may be made available
3633+
under the terms of such Secondary Licenses, and
3634+
3635+
b) a copy of this Agreement must be included with each copy of
3636+
the Program.
3637+
3638+
3.3 Contributors may not remove or alter any copyright, patent,
3639+
trademark, attribution notices, disclaimers of warranty, or limitations
3640+
of liability ("notices") contained within the Program from any copy of
3641+
the Program which they Distribute, provided that Contributors may add
3642+
their own appropriate notices.
3643+
3644+
4. COMMERCIAL DISTRIBUTION
3645+
3646+
Commercial distributors of software may accept certain responsibilities
3647+
with respect to end users, business partners and the like. While this
3648+
license is intended to facilitate the commercial use of the Program,
3649+
the Contributor who includes the Program in a commercial product
3650+
offering should do so in a manner which does not create potential
3651+
liability for other Contributors. Therefore, if a Contributor includes
3652+
the Program in a commercial product offering, such Contributor
3653+
("Commercial Contributor") hereby agrees to defend and indemnify every
3654+
other Contributor ("Indemnified Contributor") against any losses,
3655+
damages and costs (collectively "Losses") arising from claims, lawsuits
3656+
and other legal actions brought by a third party against the Indemnified
3657+
Contributor to the extent caused by the acts or omissions of such
3658+
Commercial Contributor in connection with its distribution of the Program
3659+
in a commercial product offering. The obligations in this section do not
3660+
apply to any claims or Losses relating to any actual or alleged
3661+
intellectual property infringement. In order to qualify, an Indemnified
3662+
Contributor must: a) promptly notify the Commercial Contributor in
3663+
writing of such claim, and b) allow the Commercial Contributor to control,
3664+
and cooperate with the Commercial Contributor in, the defense and any
3665+
related settlement negotiations. The Indemnified Contributor may
3666+
participate in any such claim at its own expense.
3667+
3668+
For example, a Contributor might include the Program in a commercial
3669+
product offering, Product X. That Contributor is then a Commercial
3670+
Contributor. If that Commercial Contributor then makes performance
3671+
claims, or offers warranties related to Product X, those performance
3672+
claims and warranties are such Commercial Contributor's responsibility
3673+
alone. Under this section, the Commercial Contributor would have to
3674+
defend claims against the other Contributors related to those performance
3675+
claims and warranties, and if a court requires any other Contributor to
3676+
pay any damages as a result, the Commercial Contributor must pay
3677+
those damages.
3678+
3679+
5. NO WARRANTY
3680+
3681+
EXCEPT AS EXPRESSLY SET FORTH IN THIS AGREEMENT, AND TO THE EXTENT
3682+
PERMITTED BY APPLICABLE LAW, THE PROGRAM IS PROVIDED ON AN "AS IS"
3683+
BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, EITHER EXPRESS OR
3684+
IMPLIED INCLUDING, WITHOUT LIMITATION, ANY WARRANTIES OR CONDITIONS OF
3685+
TITLE, NON-INFRINGEMENT, MERCHANTABILITY OR FITNESS FOR A PARTICULAR
3686+
PURPOSE. Each Recipient is solely responsible for determining the
3687+
appropriateness of using and distributing the Program and assumes all
3688+
risks associated with its exercise of rights under this Agreement,
3689+
including but not limited to the risks and costs of program errors,
3690+
compliance with applicable laws, damage to or loss of data, programs
3691+
or equipment, and unavailability or interruption of operations.
3692+
3693+
6. DISCLAIMER OF LIABILITY
3694+
3695+
EXCEPT AS EXPRESSLY SET FORTH IN THIS AGREEMENT, AND TO THE EXTENT
3696+
PERMITTED BY APPLICABLE LAW, NEITHER RECIPIENT NOR ANY CONTRIBUTORS
3697+
SHALL HAVE ANY LIABILITY FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
3698+
EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING WITHOUT LIMITATION LOST
3699+
PROFITS), HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
3700+
CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
3701+
ARISING IN ANY WAY OUT OF THE USE OR DISTRIBUTION OF THE PROGRAM OR THE
3702+
EXERCISE OF ANY RIGHTS GRANTED HEREUNDER, EVEN IF ADVISED OF THE
3703+
POSSIBILITY OF SUCH DAMAGES.
3704+
3705+
7. GENERAL
3706+
3707+
If any provision of this Agreement is invalid or unenforceable under
3708+
applicable law, it shall not affect the validity or enforceability of
3709+
the remainder of the terms of this Agreement, and without further
3710+
action by the parties hereto, such provision shall be reformed to the
3711+
minimum extent necessary to make such provision valid and enforceable.
3712+
3713+
If Recipient institutes patent litigation against any entity
3714+
(including a cross-claim or counterclaim in a lawsuit) alleging that the
3715+
Program itself (excluding combinations of the Program with other software
3716+
or hardware) infringes such Recipient's patent(s), then such Recipient's
3717+
rights granted under Section 2(b) shall terminate as of the date such
3718+
litigation is filed.
3719+
3720+
All Recipient's rights under this Agreement shall terminate if it
3721+
fails to comply with any of the material terms or conditions of this
3722+
Agreement and does not cure such failure in a reasonable period of
3723+
time after becoming aware of such noncompliance. If all Recipient's
3724+
rights under this Agreement terminate, Recipient agrees to cease use
3725+
and distribution of the Program as soon as reasonably practicable.
3726+
However, Recipient's obligations under this Agreement and any licenses
3727+
granted by Recipient relating to the Program shall continue and survive.
3728+
3729+
Everyone is permitted to copy and distribute copies of this Agreement,
3730+
but in order to avoid inconsistency the Agreement is copyrighted and
3731+
may only be modified in the following manner. The Agreement Steward
3732+
reserves the right to publish new versions (including revisions) of
3733+
this Agreement from time to time. No one other than the Agreement
3734+
Steward has the right to modify this Agreement. The Eclipse Foundation
3735+
is the initial Agreement Steward. The Eclipse Foundation may assign the
3736+
responsibility to serve as the Agreement Steward to a suitable separate
3737+
entity. Each new version of the Agreement will be given a distinguishing
3738+
version number. The Program (including Contributions) may always be
3739+
Distributed subject to the version of the Agreement under which it was
3740+
received. In addition, after a new version of the Agreement is published,
3741+
Contributor may elect to Distribute the Program (including its
3742+
Contributions) under the new version.
3743+
3744+
Except as expressly stated in Sections 2(a) and 2(b) above, Recipient
3745+
receives no rights or licenses to the intellectual property of any
3746+
Contributor under this Agreement, whether expressly, by implication,
3747+
estoppel or otherwise. All rights in the Program not expressly granted
3748+
under this Agreement are reserved. Nothing in this Agreement is intended
3749+
to be enforceable by any entity that is not a Contributor or Recipient.
3750+
No third-party beneficiary rights are created under this Agreement.
3751+
3752+
Exhibit A - Form of Secondary Licenses Notice
3753+
3754+
"This Source Code may also be made available under the following
3755+
Secondary Licenses when the conditions for such availability set forth
3756+
in the Eclipse Public License, v. 2.0 are satisfied: {name license(s),
3757+
version(s), and exceptions or additional permissions here}."
3758+
3759+
Simply including a copy of this Agreement, including this Exhibit A
3760+
is not sufficient to license the Source Code under Secondary Licenses.
3761+
3762+
If it is not possible or desirable to put the notice in a particular
3763+
file, then You may include the notice in a location (such as a LICENSE
3764+
file in a relevant directory) where a recipient would be likely to
3765+
look for such a notice.
3766+
3767+
You may add additional accurate notices of copyright ownership.

NOTICE

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -78,6 +78,7 @@ This software includes third party software subject to the following copyrights:
7878
- llhttp - Copyright Fedor Indutny, 2018.
7979
- benchmark - Copyright 2015 Google Inc.
8080
- llama.cpp - Copyright (c) 2023-2024 The ggml authors
81+
- Eclipse Tahu - Copyright (c) 2015, 2018 Cirrus Link Solutions and others
8182

8283
The licenses for these third party components are included in LICENSE.txt
8384

0 commit comments

Comments
 (0)