Consolidate stream processing pages (#2150)

* random typo

Signed-off-by: Alexa Kreizinger <alexakreizinger@gmail.com>

* remove/consolidate unnecessary pages

Signed-off-by: Alexa Kreizinger <alexakreizinger@gmail.com>

* remove unnecessary photo file

Signed-off-by: Alexa Kreizinger <alexakreizinger@gmail.com>

* move/rename pages

Signed-off-by: Alexa Kreizinger <alexakreizinger@gmail.com>

* update summary.md

Signed-off-by: Alexa Kreizinger <alexakreizinger@gmail.com>

* add redirects/replace hyperlinks

Signed-off-by: Alexa Kreizinger <alexakreizinger@gmail.com>

---------

Signed-off-by: Alexa Kreizinger <alexakreizinger@gmail.com>

Author: alexakreizinger

.gitbook.yaml

@@ -85,3 +85,9 @@ redirects:
 
     # Other
     concepts/buffering: ./pipeline/buffering.md
+    stream-processing/changelog: ./stream-processing/overview.md
+    stream-processing/introduction: ./stream-processing/overview.md
+    stream-processing/get-started: ./stream-processing/overview.md
+    stream-processing/getting-started/fluent-bit-sql: ./stream-processing/fluent-bit-sql.md
+    stream-processing/getting-started/check-keys-null-values: ./stream-processing/check-keys-null-values.md
+    stream-processing/getting-started/hands-on: stream-processing/tutorial.md

.gitbook/assets/stream_processor.png

@@ -27,7 +27,7 @@ description: High Performance Telemetry Agent for Logs, Metrics and Traces
     - Wasm: [Wasm Filter Plugins](development/wasm-filter-plugins.md) or [Wasm Input Plugins](development/wasm-input-plugins.md)
     - Write [Filters in Lua](pipeline/filters/lua.md) or [Output plugins in Golang](development/golang-output-plugins.md)
 - [Monitoring](administration/monitoring.md): Expose internal metrics over HTTP in JSON and [Prometheus](https://prometheus.io/) format
-- [Stream Processing](stream-processing/introduction.md): Perform data selection and transformation using basic SQL queries
+- [Stream Processing](stream-processing/overview.md): Perform data selection and transformation using basic SQL queries
   - Create new streams of data using query results
   - Aggregation windows
   - Data analysis and prediction: Time series forecasting

README.md

@@ -46,23 +46,23 @@
   * [YAML configuration](administration/configuring-fluent-bit/yaml.md)
     * [Configuration file](administration/configuring-fluent-bit/yaml/configuration-file.md)
     * [Environment variables](administration/configuring-fluent-bit/yaml/environment-variables-section.md)
-    * [Includes](administration/configuring-fluent-bit/yaml/includes-section.md) 
+    * [Includes](administration/configuring-fluent-bit/yaml/includes-section.md)
     * [Service](administration/configuring-fluent-bit/yaml/service-section.md)
     * [Parsers](administration/configuring-fluent-bit/yaml/parsers-section.md)
     * [Multiline parsers](administration/configuring-fluent-bit/yaml/multiline-parsers-section.md)
     * [Pipeline](administration/configuring-fluent-bit/yaml/pipeline-section.md)
     * [Plugins](administration/configuring-fluent-bit/yaml/plugins-section.md)
     * [Upstream servers](administration/configuring-fluent-bit/yaml/upstream-servers-section.md)
   * [Classic mode](administration/configuring-fluent-bit/classic-mode.md)
-    * [Configuration file](administration/configuring-fluent-bit/classic-mode/configuration-file.md) 
+    * [Configuration file](administration/configuring-fluent-bit/classic-mode/configuration-file.md)
     * [Commands](administration/configuring-fluent-bit/classic-mode/commands.md)
     * [Format and schema](administration/configuring-fluent-bit/classic-mode/format-schema.md)
-    * [Record accessor syntax](administration/configuring-fluent-bit/classic-mode/record-accessor.md) 
+    * [Record accessor syntax](administration/configuring-fluent-bit/classic-mode/record-accessor.md)
     * [Upstream servers](administration/configuring-fluent-bit/classic-mode/upstream-servers.md)
     * [Variables](administration/configuring-fluent-bit/classic-mode/variables.md)
   * [Multiline parsing](administration/configuring-fluent-bit/multiline-parsing.md)
   * [Unit Sizes](administration/configuring-fluent-bit/unit-sizes.md)
-* [AWS credentials](administration/aws-credentials.md) 
+* [AWS credentials](administration/aws-credentials.md)
 * [Backpressure](administration/backpressure.md)
 * [Buffering and storage](administration/buffering-and-storage.md)
 * [Hot reload](administration/hot-reload.md)
@@ -73,7 +73,7 @@
 * [Networking](administration/networking.md)
 * [Performance tips](administration/performance.md)
 * [Scheduling and retries](administration/scheduling-and-retries.md)
-* [TLS](administration/transport-security.md) 
+* [TLS](administration/transport-security.md)
 * [Troubleshooting](administration/troubleshooting.md)
 
 ## Local testing
@@ -225,13 +225,10 @@
 
 ## Stream processing
 
-* [Introduction to stream processing](stream-processing/introduction.md)
 * [Overview](stream-processing/overview.md)
-* [Changelog](stream-processing/changelog.md)
-* [Get started](stream-processing/get-started.md)
-  * [Fluent Bit and SQL](stream-processing/getting-started/fluent-bit-sql.md)
-  * [Check keys and null values](stream-processing/getting-started/check-keys-null-values.md)
-  * [Tutorial](stream-processing/getting-started/hands-on.md)
+* [Fluent Bit and SQL](stream-processing/fluent-bit-sql.md)
+* [Check keys and null values](stream-processing/check-keys-null-values.md)
+* [Tutorial](stream-processing/tutorial.md)
 
 ## Fluent Bit for developers
 

SUMMARY.md

@@ -27,7 +27,7 @@ The `Service` section defines global properties of the service. The following ke
 | `log_level`       | Set the logging verbosity level. Allowed values are: `off`, `error`, `warn`, `info`, `debug`, and `trace`. Values are cumulative. If `debug` is set, it will include `error`, `warning`, `info`, and `debug`. Trace mode is only available if Fluent Bit was built with the _`WITH_TRACE`_ option enabled. | `info` |
 | `parsers_file`    | Path for a `parsers` configuration file. Multiple `Parsers_File` entries can be defined within the section. | _none_ |
 | `plugins_file`    | Path for a `plugins` configuration file. A `plugins` configuration file defines paths for external plugins. [See an example](https://github.com/fluent/fluent-bit/blob/master/conf/plugins.conf). | _none_ |
-| `streams_file`    | Path for the Stream Processor configuration file. [Learn more about Stream Processing configuration](../../../stream-processing/introduction.md). | _none_|
+| `streams_file`    | Path for the Stream Processor configuration file. [Learn more about Stream Processing configuration](../../../stream-processing/overview.md). | _none_|
 | `http_server`     | Enable the built-in HTTP Server. | `Off` |
 | `http_listen`     | Set listening interface for HTTP Server when it's enabled. | `0.0.0.0` |
 | `http_port`       | Set TCP Port for the HTTP Server. | `2020` |

administration/configuring-fluent-bit/classic-mode/configuration-file.md

@@ -68,7 +68,7 @@ The `service` section defines the global properties of the service. The Service
 | `log_level` | Set the logging verbosity level. Allowed values are: `off`, `error`, `warn`, `info`, `debug`, and `trace`. Values are accumulative. For example, if `debug` is set, it will include `error`, `warning`, `info`, and `debug`. `trace` mode is only available if Fluent Bit was built with the `WITH_TRACE` option enabled. | `info` |
 | `parsers_file` | Path for a file that defines custom parsers. Only a single entry is supported. | _none_ |
 | `plugins_file` | Path for a `plugins` configuration file. A `plugins` configuration file allows the definition of paths for external plugins; for an example, [see here](https://github.com/fluent/fluent-bit/blob/master/conf/plugins.conf). | _none_ |
-| `streams_file` | Path for the Stream Processor configuration file. Learn more about [Stream Processing configuration](../../../stream-processing/introduction.md). | _none_ |
+| `streams_file` | Path for the Stream Processor configuration file. Learn more about [Stream Processing configuration](../../../stream-processing/overview.md). | _none_ |
 | `http_server` | Enable built-in HTTP server. | `Off` |
 | `http_listen` | Set listening interface for HTTP server when it's enabled. | `0.0.0.0` |
 | `http_port` | Set TCP Port for the HTTP server | `2020` |

administration/configuring-fluent-bit/yaml/configuration-file.md

@@ -12,7 +12,7 @@ The `service` section defines global properties of the service. The available co
 | `log_level` | Sets the logging verbosity level. Allowed values are: `off`, `error`, `warn`, `info`, `debug`, and `trace`. Values are cumulative. If `debug` is set, it will include `error`, `warn`, `info`, and `debug`. Trace mode is only available if Fluent Bit was built with the _`WITH_TRACE`_ option enabled.                                                                                                                                             | `info` |
 | `parsers_file` | Path for a `parsers` configuration file. Multiple `parsers_file` entries can be defined within the section. However, with the new YAML configuration schema, defining parsers using this key is now optional. Parsers can be declared directly in the `parsers` section of your YAML configuration, offering a more streamlined and integrated approach.                                                                                             | _none_ |
 | `plugins_file` | Path for a `plugins` configuration file. This file specifies the paths to external plugins (.so files) that Fluent Bit can load at runtime. With the new YAML schema, the `plugins_file` key is optional. External plugins can now be referenced directly within the `plugins` section, simplifying the plugin management process. [See an example](https://github.com/fluent/fluent-bit/blob/master/conf/plugins.conf).                             | _none_ |
-| `streams_file` | Path for the Stream Processor configuration file. This file defines the rules and operations for stream processing within Fluent Bit. The `streams_file` key is optional, as Stream Processor configurations can be defined directly in the `streams` section of the YAML schema. This flexibility allows for easier and more centralized configuration. [Learn more about Stream Processing configuration](../../../stream-processing/introduction.md). | _none_ |
+| `streams_file` | Path for the Stream Processor configuration file. This file defines the rules and operations for stream processing within Fluent Bit. The `streams_file` key is optional, as Stream Processor configurations can be defined directly in the `streams` section of the YAML schema. This flexibility allows for easier and more centralized configuration. [Learn more about Stream Processing configuration](../../../stream-processing/overview.md). | _none_ |
 | `http_server` | Enables the built-in HTTP Server.                                                                                                                                                                                                                                                                                                                                                                                                                    | `off` |
 | `http_listen` | Sets the listening interface for the HTTP Server when it's enabled.                                                                                                                                                                                                                                                                                                                                                                                  | `0.0.0.0` |
 | `http_port` | Sets the TCP port for the HTTP Server.                                                                                                                                                                                                                                                                                                                                                                                                               | `2020` |

administration/configuring-fluent-bit/yaml/service-section.md

@@ -6,7 +6,7 @@ The build process requires the following components:
 
 - Compiler: GCC or clang
 - CMake
-- Flex and Bison: Required for [Stream Processor](https://docs.fluentbit.io/manual/stream-processing/introduction) or [Record Accessor](https://docs.fluentbit.io/manual/administration/configuring-fluent-bit/classic-mode/record-accessor)
+- Flex and Bison: Required for [Stream Processor](https://docs.fluentbit.io/manual/stream-processing/overview) or [Record Accessor](https://docs.fluentbit.io/manual/administration/configuring-fluent-bit/classic-mode/record-accessor)
 - Libyaml development headers and libraries
 
 Core has no other dependencies. Some features depend on third-party components. For example, output plugins with special backend libraries like Kafka include those libraries in the main source code repository.

installation/requirements.md

@@ -53,7 +53,7 @@ In your main configuration file append the following:
 pipeline:
 
   outputs:
-    - name: kinesis_steams
+    - name: kinesis_streams
       match: '*'
       region: us-east-1
       stream: my-stream