refactor: carry out multiple refactors and create new code comments

Author: CoolSpring8

README.md

@@ -1,22 +1,43 @@
 # go-lolhtml
 
-![GitHub Workflow Status](https://img.shields.io/github/workflow/status/coolspring8/go-lolhtml/Go) ![Codecov](https://img.shields.io/codecov/c/github/coolspring8/go-lolhtml) [![Go Report Card](https://goreportcard.com/badge/github.com/coolspring8/go-lolhtml)](https://goreportcard.com/report/github.com/coolspring8/go-lolhtml) [![PkgGoDev](https://pkg.go.dev/badge/github.com/coolspring8/go-lolhtml)](https://pkg.go.dev/github.com/coolspring8/go-lolhtml)
+![GitHub Workflow Status](https://img.shields.io/github/workflow/status/coolspring8/go-lolhtml/Go) [![codecov](https://codecov.io/gh/CoolSpring8/go-lolhtml/branch/main/graph/badge.svg)](https://codecov.io/gh/CoolSpring8/go-lolhtml) [![Go Report Card](https://goreportcard.com/badge/github.com/coolspring8/go-lolhtml)](https://goreportcard.com/report/github.com/coolspring8/go-lolhtml) [![PkgGoDev](https://pkg.go.dev/badge/github.com/coolspring8/go-lolhtml)](https://pkg.go.dev/github.com/coolspring8/go-lolhtml)
 
-Go bindings for the Rust library [cloudflare/lol-html](https://github.com/cloudflare/lol-html/), the *Low Output Latency streaming HTML rewriter/parser with CSS-selector based API*, talking via cgo.
+Go bindings for the Rust crate [cloudflare/lol-html](https://github.com/cloudflare/lol-html/), the *Low Output Latency streaming HTML rewriter/parser with CSS-selector based API*, talking via cgo.
 
-**Status:** All abilities provided by C-API implemented, except for customized user data in handlers. Tests are partially covered. The code is at its early stage and the API is therefore subject to change. If you have any ideas on how API can be better structured, feel free to open a PR or an issue.
+**Status:** 
+
+**All abilities provided by lol_html's c-api are available**, except for customized user data in handlers. The original tests included in c-api package have also been translated to examine this binding's functionality.
+
+The code is at its early stage and **breaking changes might be introduced**. If you have any ideas on how the public API can be better structured, feel free to open a PR or an issue.
+
+   * [go-lolhtml](#go-lolhtml)
+      * [Installation](#installation)
+      * [Features](#features)
+      * [Getting Started](#getting-started)
+      * [Examples](#examples)
+      * [Documentation](#documentation)
+      * [Other Bindings](#other-bindings)
+      * [Versioning](#versioning)
+      * [Help Wanted!](#help-wanted)
+      * [License](#license)
+      * [Disclaimer](#disclaimer)
 
 ## Installation
 
-For Linux/macOS/Windows x86_64 platforms, installation is as simple as a single `go get`:
+For Linux/macOS/Windows x86_64 platform users, installation is as simple as a single `go get` command:
 
 ```shell
 $ go get github.com/coolspring8/go-lolhtml
 ```
 
-There is no need for you to install Rust. That's because lol-html could be prebuilt into static libraries, stored and shipped in `/build` folder, so that cgo can handle other matters naturally and smoothly.
+Installing Rust is not a necessary step. That's because lol-html could be prebuilt into static libraries, stored and shipped in `/build` folder, so that cgo can handle other compilation matters naturally and smoothly, without intervention.
 
-For other platforms, you'll have to compile it yourself.
+For other platforms, you will have to compile it yourself.
+
+## Features
+
+- Fast: A Go (cgo) wrapper built around the highly-optimized Rust HTML parsing crate lol_html.
+- Easy to use: Utilizing Go's idiomatic I/O methods, [lolhtml.Writer](https://pkg.go.dev/github.com/coolspring8/go-lolhtml#Writer) implements [io.Writer](https://golang.org/pkg/io/#Writer) interface.
 
 ## Getting Started
 
@@ -27,16 +48,18 @@ package main
 
 import (
 	"bytes"
-	"github.com/coolspring8/go-lolhtml"
 	"io"
 	"log"
 	"os"
+
+	"github.com/coolspring8/go-lolhtml"
 )
 
 func main() {
 	chunk := []byte("Hello, <span>World</span>!")
 	r := bytes.NewReader(chunk)
 	w, err := lolhtml.NewWriter(
+		// output to stdout
 		os.Stdout,
 		&lolhtml.Handlers{
 			ElementContentHandler: []lolhtml.ElementContentHandler{
@@ -56,26 +79,43 @@ func main() {
 	if err != nil {
 		log.Fatal(err)
 	}
-	defer w.Free()
 
+	// copy from the bytes reader to lolhtml writer
 	_, err = io.Copy(w, r)
 	if err != nil {
 		log.Fatal(err)
 	}
-    
-	err = w.End()
+
+	// explicitly close the writer and flush the remaining content
+	err = w.Close()
 	if err != nil {
 		log.Fatal(err)
 	}
 	// Output: Hello, <span>LOL-HTML</span>!
 }
 ```
 
-The above program takes the chunk `Hello, <span>World</span>!` as input, is configured to rewrite all texts in `span` tags to "LOL-HTML" and prints the result to standard output.
+The above program creates a new Writer configured to rewrite all texts in `span` tags to "LOL-HTML". It takes the chunk `Hello, <span>World</span>!` as input, and prints the result to standard output.
 
 And the result is `Hello, <span>LOL-HTML</span>!` .
 
-For more examples, please see the `/examples` directory.
+## Examples
+
+example_test.go contains two examples.
+
+For more detailed examples, please visit the `/examples` subdirectory.
+
+- defer-scripts
+
+  Usage: curl -NL https://git.io/JeOSZ | go run main.go
+
+- mixed-content-rewriter
+
+  Usage: curl -NL https://git.io/JeOSZ | go run main.go
+
+- web-scraper
+
+  A ported Go version of https://web.scraper.workers.dev/.
 
 ## Documentation
 
@@ -86,6 +126,14 @@ Available at [pkg.go.dev](https://pkg.go.dev/github.com/coolspring8/go-lolhtml).
 - Rust (native), C, JavaScript - [cloudflare/lol-html](https://github.com/cloudflare/lol-html/)
 - Lua - [jdesgats/lua-lolhtml](https://github.com/jdesgats/lua-lolhtml/)
 
+## Versioning
+
+This package does not really follow [Semantic Versioning](https://semver.org/). The current strategy is to follow lol_html's major and minor version, and the patch version number is reserved for this binding's updates, for Go Modul to upgrade correctly.
+
+## Help Wanted!
+
+There are a few interesting things at [Projects](https://github.com/coolspring8/go-lolhtml/projects/1) panel that I have considered but is not yet implemented. Other contributions and suggestions are also welcome!
+
 ## License
 
 BSD 3-Clause "New" or "Revised" License

attribute.go

@@ -6,24 +6,33 @@ package lolhtml
 */
 import "C"
 
-// AttributeIterator cannot be iterated by "range" syntax. You should use AttributeIterator.Next() instead.
+// AttributeIterator can be used to iterate over all attributes of an element. The only way to
+// get an AttributeIterator is by calling AttributeIterator() on an Element. Note the "range" syntax is not
+// applicable here, use AttributeIterator.Next() instead.
 type AttributeIterator C.lol_html_attributes_iterator_t
+
+// Attribute represents an HTML element attribute. Obtained by calling Next() on an AttributeIterator.
 type Attribute C.lol_html_attribute_t
 
+// Free frees the memory held by the AttributeIterator.
 func (ai *AttributeIterator) Free() {
 	C.lol_html_attributes_iterator_free((*C.lol_html_attributes_iterator_t)(ai))
 }
 
+// Next advances the iterator and returns next attribute.
+// Returns nil if the iterator has been exhausted.
 func (ai *AttributeIterator) Next() *Attribute {
 	return (*Attribute)(C.lol_html_attributes_iterator_next((*C.lol_html_attributes_iterator_t)(ai)))
 }
 
+// Name returns the name of the attribute.
 func (a *Attribute) Name() string {
 	nameC := (str)(C.lol_html_attribute_name_get((*C.lol_html_attribute_t)(a)))
 	defer nameC.Free()
 	return nameC.String()
 }
 
+// Value returns the value of the attribute.
 func (a *Attribute) Value() string {
 	valueC := (str)(C.lol_html_attribute_value_get((*C.lol_html_attribute_t)(a)))
 	defer valueC.Free()

benchmark_test.go

@@ -231,11 +231,10 @@ func BenchmarkNewWriter(b *testing.B) {
 						b.Fatal(err)
 					}
 
-					err = w.End()
+					err = w.Close()
 					if err != nil {
 						b.Fatal(err)
 					}
-					w.Free()
 				}
 			})
 		}

builder.go

@@ -130,7 +130,7 @@ func (rb *rewriterBuilder) Build(sink OutputSink, config Config) (*rewriter, err
 	)
 	if r != nil {
 		rb.built = true
-		return &rewriter{rw: r, pointers: rb.pointers}, nil
+		return &rewriter{rewriter: r, pointers: rb.pointers}, nil
 	}
 	return nil, getError()
 }

comment.go

@@ -7,16 +7,21 @@ package lolhtml
 import "C"
 import "unsafe"
 
+// Comment represents an HTML comment.
 type Comment C.lol_html_comment_t
 
+// CommentHandlerFunc is a callback handler function to do something with a Comment.
+// Expected to return a RewriterDirective as instruction to continue or stop.
 type CommentHandlerFunc func(*Comment) RewriterDirective
 
+// Text returns the comment's text.
 func (c *Comment) Text() string {
 	textC := (str)(C.lol_html_comment_text_get((*C.lol_html_comment_t)(c)))
 	defer textC.Free()
 	return textC.String()
 }
 
+// SetText sets the comment's text and returns an error if there is one.
 func (c *Comment) SetText(text string) error {
 	textC := C.CString(text)
 	defer C.free(unsafe.Pointer(textC))
@@ -36,18 +41,18 @@ const (
 	commentReplace
 )
 
-func (c *Comment) alter(content string, alter commentAlter, isHtml bool) error {
+func (c *Comment) alter(content string, alter commentAlter, isHTML bool) error {
 	contentC := C.CString(content)
 	defer C.free(unsafe.Pointer(contentC))
 	contentLen := len(content)
 	var errCode C.int
 	switch alter {
 	case commentInsertBefore:
-		errCode = C.lol_html_comment_before((*C.lol_html_comment_t)(c), contentC, C.size_t(contentLen), C.bool(isHtml))
+		errCode = C.lol_html_comment_before((*C.lol_html_comment_t)(c), contentC, C.size_t(contentLen), C.bool(isHTML))
 	case commentInsertAfter:
-		errCode = C.lol_html_comment_after((*C.lol_html_comment_t)(c), contentC, C.size_t(contentLen), C.bool(isHtml))
+		errCode = C.lol_html_comment_after((*C.lol_html_comment_t)(c), contentC, C.size_t(contentLen), C.bool(isHTML))
 	case commentReplace:
-		errCode = C.lol_html_comment_replace((*C.lol_html_comment_t)(c), contentC, C.size_t(contentLen), C.bool(isHtml))
+		errCode = C.lol_html_comment_replace((*C.lol_html_comment_t)(c), contentC, C.size_t(contentLen), C.bool(isHTML))
 	default:
 		panic("not implemented")
 	}
@@ -57,34 +62,69 @@ func (c *Comment) alter(content string, alter commentAlter, isHtml bool) error {
 	return getError()
 }
 
+// InsertBeforeAsText inserts the given content before the comment.
+//
+// The rewriter will HTML-escape the content before insertion:
+//
+// `<` will be replaced with `&lt;`
+//
+// `>` will be replaced with `&gt;`
+//
+// `&` will be replaced with `&amp;`
 func (c *Comment) InsertBeforeAsText(content string) error {
 	return c.alter(content, commentInsertAfter, false)
 }
 
-func (c *Comment) InsertBeforeAsHtml(content string) error {
+// InsertBeforeAsHTML inserts the given content before the comment.
+// The content is inserted as is.
+func (c *Comment) InsertBeforeAsHTML(content string) error {
 	return c.alter(content, commentInsertBefore, true)
 }
 
+// InsertAfterAsText inserts the given content before the comment.
+//
+// The rewriter will HTML-escape the content before insertion:
+//
+// `<` will be replaced with `&lt;`
+//
+// `>` will be replaced with `&gt;`
+//
+// `&` will be replaced with `&amp;`
 func (c *Comment) InsertAfterAsText(content string) error {
 	return c.alter(content, commentInsertAfter, false)
 }
 
-func (c *Comment) InsertAfterAsHtml(content string) error {
+// InsertAfterAsHTML inserts the given content before the comment.
+// The content is inserted as is.
+func (c *Comment) InsertAfterAsHTML(content string) error {
 	return c.alter(content, commentInsertAfter, true)
 }
 
+// ReplaceAsText replace the comment with the supplied content.
+//
+// The rewriter will HTML-escape the content:
+//
+// `<` will be replaced with `&lt;`
+//
+// `>` will be replaced with `&gt;`
+//
+// `&` will be replaced with `&amp;`
 func (c *Comment) ReplaceAsText(content string) error {
 	return c.alter(content, commentReplace, false)
 }
 
-func (c *Comment) ReplaceAsHtml(content string) error {
+// ReplaceAsHTML replace the comment with the supplied content.
+// The content is kept as is.
+func (c *Comment) ReplaceAsHTML(content string) error {
 	return c.alter(content, commentReplace, true)
 }
 
+// Remove removes the comment.
 func (c *Comment) Remove() {
 	C.lol_html_comment_remove((*C.lol_html_comment_t)(c))
 }
 
+// IsRemoved returns whether the comment is removed or not.
 func (c *Comment) IsRemoved() bool {
 	return (bool)(C.lol_html_comment_is_removed((*C.lol_html_comment_t)(c)))
 }