Skip to content

New batch of quickstart edits #257

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
VB255047 merged 1 commit into from
Jul 16, 2026
Merged

New batch of quickstart edits #257

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
150 changes: 90 additions & 60 deletions quickstarts/analyze-data/sto.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,32 +1,31 @@
---
sidebar_position: 4
id: sto
title: Run scripts on Vantage
author: Adam Tworkiewicz
email: adam.tworkiewicz@teradata.com
page_last_update: September 7th, 2021
title: Run scripts on Teradata
author: Adam Tworkiewicz, Vidhan Bhonsle
email: developer.relations@teradata.com
page_last_update: July 13th, 2026
description: Run Applications on Teradata - use Script Table Operator to run applications on your data without data movement.
keywords: [data warehouses, compute storage separation, teradata, vantage, script table operator, cloud data platform, object storage, business intelligence, enterprise analytics]
keywords: [data warehouses, compute storage separation, teradata, script table operator, cloud data platform, object storage, business intelligence, enterprise analytics]
---

import TrialDocsNote from '../_partials/teradata_trial.mdx'
import CommunityLink from '../_partials/community_link.mdx'

# Run scripts on Vantage
# Run scripts on Teradata

## Overview

Sometimes, you need to apply complex logic to your data that can't be easily expressed in SQL. One option is to wrap your logic in a User Defined Function (UDF). What if you already have this logic coded in a language that is not supported by UDF? Script Table Operator is a Vantage feature that allows you to bring your logic to the data and run it on Vantage. The advantage of this approach is that you don't have to retrieve data from Vantage to operate on it. Also, by running your data applications on Vantage, you leverage its parallel nature. You don't have to think how your applications will scale. You can let Vantage take care of it.
Sometimes, you need to apply complex logic to your data that can't be easily expressed in SQL. One option is to wrap your logic in a User Defined Function (UDF). What if you already have this logic coded in a language that is not supported by UDF? Script Table Operator is a Teradata feature that allows you to bring your logic to the data and run it on Teradata. The advantage of this approach is that you don't have to retrieve data from Teradata to operate on it. Also, by running your data applications on Teradata, you leverage its parallel nature. You don't have to think how your applications will scale. You can let Teradata take care of it.

## Prerequisites

You need access to a Teradata Vantage instance.
You need access to a Teradata instance.

<TrialDocsNote />

## Hello World

Let's start with something simple. What if you wanted the database to print "Hello World"?
Let's start with something simple. What if we wanted the database to print "Hello World"?

```sql
SELECT *
Expand All @@ -36,15 +35,17 @@ FROM
RETURNS ('Message varchar(512)'));
```

Here is what I've got:
Here is what we've got:
```sql
Message
------------
Hello World!
Hello World!
Hello World!
Hello World!
```

Let's analyze what just happened here. The SQL includes `echo Hello World!`. This is a Bash command. OK, so now we know how to run Bash commands. But why did we get 2 rows and not one? That's because our simple script was run once on each AMP and I happen to have 2 AMPs:
Let's analyze what just happened here. The SQL includes `echo Hello World!`, which is a Bash command. The script runs once on each AMP, so the number of returned rows depends on the number of AMPs in your Teradata system.

```sql
-- Teradata magic that returns the number of AMPs in a system
Expand All @@ -55,59 +56,72 @@ Returns:
```sql
number_of_amps
--------------
2
4
```

This simple script demonstrates the idea behind the Script Table Operator (STO). You provide your script and the database runs it in parallel, once for each AMP. This is an attractive model in case you have transformation logic in a script and a lot of data to process. Normally, you would need to build concurrency into your application. By letting STO do it, you let Teradata select the right concurrency level for your data.

## Supported languages

OK, so we did `echo` in Bash but Bash is hardly a productive environment to express complex logic. What other languages are supported then? The good news is that any binary that can run on Vantage nodes can be used in STO. Remember, that the binary and all its dependencies need to be installed on all your Vantage nodes. In practice, it means that your options will be limited to what your administrator is willing and able to maintain on your servers. Python is a very popular choice.
We used `echo` in Bash, but Bash is hardly a productive environment for expressing complex logic. What other languages are supported? The good news is that any binary that can run on Teradata nodes can be used in STO. Remember that the binary and all its dependencies must be installed on all your Teradata nodes. In practice, the available options depend on what your administrator is willing and able to maintain on the servers. Python is a popular choice.

## Uploading scripts

Ok, Hello World is super exciting, but what if we have existing logic in a large file. Surely, you don't want to paste your entire script and escape quotes in an SQL query. We solve the script upload issue with the User Installed Files (UIF) feature.
`Hello World` is useful for testing, but what if you have existing logic in a larger file? You likely do not want to paste the entire script and escape quotes in an SQL query. The User Installed Files (UIF) feature solves the script upload issue.

Say you have `helloworld.py` script with the following content:

```bash
print("Hello World!")
```

Let's assume the script is on your local machine at `/tmp/helloworld.py`.
Create this file on your local machine. On Linux/macOS, place it at `/tmp/helloworld.py`. On Windows, use a path like `C:\\Temp\\helloworld.py`.

First, we need to setup permissions in Vantage. We are going to do this using a new database to keep it clean.
First, we need to setup permissions in Teradata. We are going to do this using a new database to keep it clean.

```sql
-- Create a new database called sto
-- Create a new database called STO
CREATE DATABASE STO
AS PERMANENT = 60e6, -- 60MB
SPOOL = 120e6; -- 120MB

-- Allow dbc user to create scripts in database STO
GRANT CREATE EXTERNAL PROCEDURE ON STO to dbc;
-- Check the current Teradata user
SELECT USER;

-- Replace <CURRENT_USER_VALUE> with the value returned by SELECT USER
GRANT CREATE EXTERNAL PROCEDURE ON STO TO <CURRENT_USER_VALUE>;
```

You can upload the script to Vantage using the following procedure call:
You can upload the script to Teradata using the following procedure call.
:::note
Adjust the path to match your client OS and where the local file is located.
:::

```python
call SYSUIF.install_file('helloworld',
'helloworld.py', 'cz!/tmp/helloworld.py');
Linux/macOS example:

```sql
CALL SYSUIF.install_file('helloworld', 'helloworld.py', 'cz!/tmp/helloworld.py');
```

Windows example:

```sql
CALL SYSUIF.install_file('helloworld', 'helloworld.py', 'cz!C:/Temp/helloworld.py');
```

Now that the script has been uploaded, you can call it like this:
Now that the script has been uploaded, you can call it like this. Note: Run this as a separate statement from the CALL above, or issue an ET/NULL if required by your client.

```sql
-- We switch to STO database
DATABASE STO
DATABASE STO;

-- We tell Vantage where to look for the script. This can be
-- We tell Teradata where to look for the script. This can be
-- any string and it will create a symbolic link to the directory
-- where our script got uploaded. By convention, we use the
-- database name.
SET SESSION SEARCHUIFDBPATH = sto;

-- We now call the script. Note, how we use a relative path that
-- We now call the script. Note how we use a relative path that
-- starts with `./sto/`, which is where SEARCHUIFDBPATH
-- is pointing.
SELECT *
Expand All @@ -122,26 +136,33 @@ Message
------------
Hello World!
Hello World!
Hello World!
Hello World!
```

That was a lot of work and we are still at Hello World. Let's try to pass some data into `SCRIPT`.

## Passing data stored in Vantage to SCRIPT
## Passing data stored in Teradata to SCRIPT

So far, we have been using `SCRIPT` operator to run standalone scripts. But the main purpose to run scripts on Vantage is to process data that is in Vantage. Let's see how we can retrieve data from Vantage and pass it to `SCRIPT`.
So far, we have been using `SCRIPT` operator to run standalone scripts. But the main purpose to run scripts on Teradata is to process data that is in Teradata. Let's see how we can retrieve data from Teradata and pass it to `SCRIPT`.

We will start with creating a table with a few rows.
We will start by creating a table with a few rows.

```sql
-- Switch to STO database.
DATABASE STO
DATABASE STO;

-- Create a table with a few urls
CREATE TABLE urls(url varchar(10000));
INS urls('https://www.google.com/finance?q=NYSE:TDC');
INS urls('http://www.ebay.com/sch/i.html?_trksid=p2050601.m570.l1313.TR0.TRC0.H0.Xteradata+merchandise&_nkw=teradata+merchandise&_sacat=0&_from=R40');
INS urls('https://www.youtube.com/results?search_query=teradata%20commercial&sm=3');
INS urls('https://www.contrivedexample.com/example?mylist=1&mylist=2&mylist=...testing');
```

Now insert the data. Insert each row in a separate statement:

```sql
INSERT INTO urls VALUES ('https://www.google.com/finance?q=NYSE:TDC');
INSERT INTO urls VALUES ('http://www.ebay.com/sch/i.html?_trksid=p2050601.m570.l1313.TR0.TRC0.H0.Xteradata+merchandise&_nkw=teradata+merchandise&_sacat=0&_from=R40');
INSERT INTO urls VALUES ('https://www.youtube.com/results?search_query=teradata%20commercial&sm=3');
INSERT INTO urls VALUES ('https://www.contrivedexample.com/example?mylist=1&mylist=2&mylist=...testing');
```

We will use the following script to parse out query parameters:
Expand All @@ -161,18 +182,27 @@ for line in sys.stdin:
print("\t".join(element))
```

Note, how the scripts assumes that urls will be fed into `stdin` one by one, line by line. Also, note how it prints results line by line, using the tab character as a delimiter between values.
Note how the script assumes that urls will be fed into `stdin` one by one, line by line. Also, note how it prints results line by line, using the tab character as a delimiter between values.

Let's install the script. Here, we assume that the script file is at `/tmp/urlparser.py` on our local machine:
```python
CALL SYSUIF.install_file('urlparser',
'urlparser.py', 'cz!/tmp/urlparser.py');
Let's install the script.

Linux/macOS example:

```sql
CALL SYSUIF.install_file('urlparser', 'urlparser.py', 'cz!/tmp/urlparser.py');
```

Windows example:

```sql
CALL SYSUIF.install_file('urlparser', 'urlparser.py', 'cz!C:/Temp/urlparser.py');
```

With the script installed, we will now retrieve data from `urls` table and feed it into the script to retrieve query parameters:

```sql
-- We inform Vantage to create a symbolic link from the UIF directory to ./sto/
SET SESSION SEARCHUIFDBPATH = sto ;
-- We inform Teradata to create a symbolic link from the UIF directory to ./sto/
SET SESSION SEARCHUIFDBPATH = sto;

SELECT *
FROM SCRIPT(
Expand All @@ -199,22 +229,24 @@ mylist |...testing

## Inserting SCRIPT output into a table

We have learned how to take data from Vantage, pass it to a script and get output. Is there an easy way to store this output in a table? Sure, there is. We can combine the select above with `CREATE TABLE` statement:
We have learned how to take data from Teradata, pass it to a script and get output. Is there an easy way to store this output in a table? Sure, there is. We can use the following pattern:

```sql
-- We inform Vantage to create a symbolic link from the UIF directory to ./sto/
SET SESSION SEARCHUIFDBPATH = sto ;
-- We inform Teradata to create a symbolic link from the UIF directory to ./sto/
SET SESSION SEARCHUIFDBPATH = sto;

CREATE MULTISET TABLE
url_params(param_key, param_value)
AS (
SELECT *
FROM SCRIPT(
ON(SELECT url FROM urls)
SCRIPT_COMMAND('python3 ./sto/urlparser.py')
RETURNS ('param_key varchar(512)', 'param_value varchar(512)'))
) WITH DATA
-- First, create the table structure
CREATE MULTISET TABLE url_params
(param_key varchar(512), param_value varchar(512))
NO PRIMARY INDEX;

-- Then insert the SCRIPT results
INSERT INTO url_params
SELECT *
FROM SCRIPT(
ON(SELECT url FROM urls)
SCRIPT_COMMAND('python3 ./sto/urlparser.py')
RETURNS ('param_key varchar(512)', 'param_value varchar(512)'));
```

Now, let's inspect the contents of `url_params` table:
Expand All @@ -241,10 +273,8 @@ mylist |...testing

## Summary

In this quick start we have learned how to run scripts against data in Vantage. We ran scripts using Script Table Operator (STO). The operator allows us to bring logic to the data. It offloads concurrency considerations to the database by running our scripts in parallel, one per AMP. All you need to do is provide a script and the database will execute it in parallel.
In this quick start, we learned how to run scripts against data in Teradata. We ran scripts using Script Table Operator (STO). The operator allows us to bring logic to the data. It offloads concurrency considerations to the database by running our scripts in parallel, one per AMP. All you need to do is provide a script and the database will execute it in parallel.

## Further reading
* [Teradata Vantage™ - SQL Operators and User-Defined Functions - SCRIPT](https://docs.teradata.com/r/9VmItX3V2Ni9Ts70HbDzVg/CBAaRxUyOdF0t1SQIuXeug)
* [R and Python Analytics with SCRIPT Table Operator](https://docs.teradata.com/v/u/Orange-Book/R-and-Python-Analytics-with-SCRIPT-Table-Operator-Orange-Book-4.3.1)

<CommunityLink />
* [Teradata SQL Operators and User-Defined Functions - SCRIPT](https://docs.teradata.com/r/9VmItX3V2Ni9Ts70HbDzVg/CBAaRxUyOdF0t1SQIuXeug)
* [R and Python Analytics with SCRIPT Table Operator](https://docs.teradata.com/v/u/Orange-Book/R-and-Python-Analytics-with-SCRIPT-Table-Operator-Orange-Book-4.3.1)
45 changes: 24 additions & 21 deletions quickstarts/create-applications/mule-dbc-example.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,33 +1,32 @@
---
sidebar_position: 3
id: mule-dbc-example
author: Adam Tworkiewicz
email: adam.tworkiewicz@teradata.com
page_last_update: August 30, 2023
description: Query Teradata Vantage from a Mule service.
keywords: [data warehouses, compute storage separation, teradata, vantage, cloud data platform, object storage, business intelligence, enterprise analytics, Mule, JDBC, microservices]
author: Adam Tworkiewicz, Vidhan Bhonsle
email: developer.relations@teradata.com
page_last_update: July 14th, 2026
description: Query Teradata from a Mule service.
keywords: [data warehouses, compute storage separation, teradata, cloud data platform, object storage, business intelligence, enterprise analytics, Mule, JDBC, microservices]
---

import TrialDocsNote from '../_partials/teradata_trial.mdx'
import CommunityLink from '../_partials/community_link.mdx'

# Query Teradata Vantage from a Mule service
# Query Teradata from a Mule service

## Overview

This example is a clone of the Mulesoft MySQL sample project.
It demonstrates how to query a Teradata database and expose results over REST API.
It demonstrates how to query Teradata and expose results over REST API.

## Prerequisites

* Mulesoft Anypoint Studio. You can download a 30-day trial from https://www.mulesoft.com/platform/studio.
* Access to a Teradata Vantage instance.
* Access to a Teradata instance.

<TrialDocsNote />

## Example service

This example Mule service takes an HTTP request, queries the Teradata Vantage database and returns results in JSON format.
This example Mule service takes an HTTP request, queries Teradata and returns results in JSON format.

![service flow](../images/flow.png)

Expand All @@ -43,7 +42,7 @@ As you can see, we are using parameterized query with reference to the value of
So if the HTTP connector receives http://localhost:8081/?lastname=Smith, the SQL query will be:

```sql
SELECT * FROM employees WHERE last_name = Smith
SELECT * FROM employees WHERE LastName = 'Smith'
```

The database connector instructs the database server to run the SQL query, retrieves the result of the query, and passes it to the Transform message processor which converts the result to JSON.
Expand All @@ -59,10 +58,10 @@ Since the HTTP connector is configured as request-response, the result is return
* Edit `src/main/mule/querying-a-teradata-database.xml`, find the Teradata connection string `jdbc:teradata://<HOST>/user=<username>,password=<password>` and replace Teradata connection parameters to match your environment.

:::note
Should your Vantage instance be accessible via ClearScape Analytics Experience, you must replace `<HOST>` with the host URL of your ClearScape Analytics Experience environment. Additionally, the 'user' and 'password' should be updated to reflect your ClearScape Analytics Environment's username and password.
Should your Teradata instance be accessible via Teradata Trial, you must replace `<HOST>` with the host URL of your Teradata Trial environment. Additionally, the 'user' and 'password' should be updated to reflect your Teradata Trial environment's username and password.
:::

* Create a sample database in your Vantage instance.
* Create a sample database in your Teradata instance.
Populate it with sample data.

```sql
Expand Down Expand Up @@ -110,6 +109,12 @@ Populate it with sample data.

* Use the directory where you cloned the git repository as the `Project Root`. Leave all other settings at their default values.

* If prompted to update the workspace, keep all the default options selected and click **Perform update**.

![Update Anypoint Studio workspace](../images/anypoint.perform.update.png)

This updates the project configuration so it is compatible with the current version of Anypoint Studio.

## Run

* Run the example application in Anypoint Studio using the `Run` menu.
Expand All @@ -122,12 +127,12 @@ You should get the following JSON response:
```json
[
{
"JoinedDate": "2004-08-01T00:00:00",
"DateOfBirth": "1980-01-05T00:00:00",
"FirstName": "Test",
"GlobalID": 101,
"DepartmentCode": 1,
"LastName": "Testowsky"
"FirstName": "Test",
"LastName": "Testowsky",
"DateOfBirth": "1980-01-05T00:00:00",
"JoinedDate": "2004-08-01T00:00:00",
"DepartmentCode": 1
}
]
```
Expand All @@ -136,6 +141,4 @@ You should get the following JSON response:

* View this [document](http://www.mulesoft.org/documentation/display/current/Database+Connector) for more information on how to configure a database connector on your machine.
* Access plain [Reference material](http://www.mulesoft.org/documentation/display/current/Database+Connector+Reference) for the Database Connector.
* Learn more about [DataSense](http://www.mulesoft.org/documentation/display/current/DataSense).

<CommunityLink />
* Learn more about [DataSense](http://www.mulesoft.org/documentation/display/current/DataSense).
Loading
Loading