Part 7: Data Manipulation - Maple Programming Help

Online Help

All Products    Maple    MapleSim


Home : Support : Online Help : Getting Started : Tutorials : MaplePortal/Tutorial7

Part 7: Data Manipulation

Go to Maple Portal     Previous Tutorial     Next Tutorial

 

Introduction

Importing and Exporting Data

Random Distributions

Statistics, Regression, and Curve Fitting

See Also

Introduction

Maple's Tutorials are designed to help you get started with Maple, learn about the key tools available in Maple, and lead you through a series of problems.

 

In Part 7: Data Manipulation, learn to import and export data. Explore Maple's tools for statistics, visualization, and data analysis.

 

To try this material on your own, start with an empty Maple document. Perform the steps described in the left column of each table below.  The results of the steps are displayed in the right column for reference.

 

Refer to Help>Quick Reference for basic getting started tips.

 

Note for non-Windows users: The keystrokes given in this document are for Windows.  There will be differences for other platforms. If you are using a different platform, see Shortcut Keys.

 

Importing and Exporting Data

Import and export data with interactive tools or with a command.  With Maple, you can import data from many formats and export data back to files.

Importing Data

Steps

Result

Using the Import Data Assistant

You can import data from a file.  Supported file formats include Excel, MATLAB, Image, Audio, Matrix Market and Delimited.

 

Example:  From the Tools>Assistants menu, choose Import Data...

 

Locate the data file ExcelData.xls.  This file is located in the data/portal subdirectory of your Maple installation.

 

Choose Excel as the file type.  

Click Next, Next, and Next.  If desired, add a name so you can refer to this data later, and click Done.

 

The data is imported as an Array.

 

Note: After completing these steps to import the data, you can double-click on the summary to browse the data.

 

Now, use Plot Builder to plot the data.

From the Context Panel for the imported data, select Plots>Plot Builder.

 

Click Plot.

30 x 2 MatrixData Type: anythingStorage: rectangularOrder: Fortran_order

Using the ExcelTools Package

 

You can also use the ExcelTools package to import and export data stored in Microsoft Excel format.

 

Example:

Import the Excel file ExcelData.xls.  

 

If a file is not located in the current directory, you need to enter the full path to the data file.  

 

In this case, the data is located in the data/portal directory of your Maple installation.  The command kernelopts(datadir) returns the location of the data directory.  Then, the cat command is used to concatenate the two strings to give the full path to the data file.

 

 

 

 

 

View the first row.

 

 

Return the number of elements in the Matrix.

 

 

 

Plot the data using plots[pointplot].

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

withExcelTools:

FileLocationcatkerneloptsdatadir,/portal/ExcelData.xls:

DataSetImportFileLocation

DataSet 30 x 2 MatrixData Type: anythingStorage: rectangularOrder: Fortran_order

(2.1.1)

DataSet1

0.0.918632608623005

(2.1.2)

ArrayNumElemsDataSet

60

(2.1.3)

plotspointplotDataSet,color=Green

Using the readdata Command

 

The readdata command reads numeric data from a text file.  The calling sequences is readdata(fileID, n), where n is the number of columns of data.

 

The output from readdata is a list of elements.

 

This command returns a list, so use list selection to view entries in Data1.  Then, plot the data.

FileLocation2catkerneloptsdatadir,/portal/DataP2.dat:

Data1readdataFileLocation2,2:

nopsData1

3938

(2.1.4)

Data11

0.008,−2.828784

(2.1.5)

Data1100

0.007901,−0.942928

(2.1.6)

plotData1,style=point,symbol=cross,color=Brown

The ImportMatrix command reads a data file just like readdata, but the output from ImportMatrix is a Matrix instead of a list.

Data2ImportMatrixFileLocation2

Data2 7876 x 2 MatrixData Type: anythingStorage: rectangularOrder: Fortran_order

(2.1.7)

plotData2,style=point,color=Green

Exporting Data

 

Steps

Result

Exporting an Excel File

 

Using ExcelTools, export to a file.  By default, the file is exported to the current directory.  To find what is the current directory, use the command currentdir().

 

 

 

As a test, import the first 10 rows back into Maple.

withExcelTools:

Data3 ImportMatrixcatkerneloptsdatadir,/portal/DataP2.dat:

ExportData3,TestFile.xls

 

 

ImportTestFile.xls,Sheet1,A1:B10

0.008−2.8287840.007999−2.8287840.007998−2.694080.007997−2.694080.007996−2.559376

(2.2.1)

Exporting a Text File

 

The writedata command writes data to a text file.  The calling sequence is writedata(fileID,data,format), where filename is the name given to the data file, data is the data itself, and format specifies how the data is to be written.  The options for format are integer, float, or string.

 

Example:

Create a list, filling in the entries with the seq command.

 

 

Write the data to a file.

 

As a test, import the data file back into Maple.

TestDataseq0.1i,0.1i2,i=0..100:

TestData3

0.2,0.04

(2.2.2)

writedataMyDataFile,TestData,float

readdataMyDataFile,2

0.,0.,0.1,0.01,0.2,0.04,0.3,0.09,0.4,0.16,0.5,0.25,0.6,0.36,0.7,0.49,0.8,0.64,0.9,0.81,1.,1.,1.1,1.21,1.2,1.44,1.3,1.69,1.4,1.96,1.5,2.25,1.6,2.56,1.7,2.89,1.8,3.24,1.9,3.61,2.,4.,2.1,4.41,2.2,4.84,2.3,5.29,2.4,5.76,2.5,6.25,2.6,6.76,2.7,7.29,2.8,7.84,2.9,8.41,3.,9.,3.1,9.61,3.2,10.24,3.3,10.89,3.4,11.56,3.5,12.25,3.6,12.96,3.7,13.69,3.8,14.44,3.9,15.21,4.,16.,4.1,16.81,4.2,17.64,4.3,18.49,4.4,19.36,4.5,20.25,4.6,21.16,4.7,22.09,4.8,23.04,4.9,24.01,5.,25.,5.1,26.01,5.2,27.04,5.3,28.09,5.4,29.16,5.5,30.25,5.6,31.36,5.7,32.49,5.8,33.64,5.9,34.81,6.,36.,6.1,37.21,6.2,38.44,6.3,39.69,6.4,40.96,6.5,42.25,6.6,43.56,6.7,44.89,6.8,46.24,6.9,47.61,7.,49.,7.1,50.41,7.2,51.84,7.3,53.29,7.4,54.76,7.5,56.25,7.6,57.76,7.7,59.29,7.8,60.84,7.9,62.41,8.,64.,8.1,65.61,8.2,67.24,8.3,68.89,8.4,70.56,8.5,72.25,8.6,73.96,8.7,75.69,8.8,77.44,8.9,79.21,9.,81.,9.1,82.81,9.2,84.64,9.3,86.49,9.4,88.36,9.5,90.25,9.6,92.16,9.7,94.09,9.8,96.04,9.9,98.01,10.,100.

(2.2.3)

 

Random Distributions

Steps

Result

To generate a random number, use the rand command.  The simplest calling sequence generates a random 12 digit positive integer.

 

The rand(a..b) calling sequence returns a procedure which then generates numbers between a and b.

 

In order to ensure that a different procedure is created every time the code is run, the randomize command is used to reset the seed for the random number generator.

 

Example:

Generate random numbers between -10 and 10.

 

 

 

 

 

 

 

 

The rand command provides a simple interface to the RandomTools package, which consists of a large collection of tools and algorithms for random number and random object generation.

 

rand

395718860534

(3.1)

randomize:

r  rand10..10:

r;

5

(3.2)

r;

5

(3.3)

r;

4

(3.4)

Maple has a built-in Statistics package that provides a large number of continuous and discrete distributions that can be used to generate random numbers. To use these distributions, load the Statistics package.

 

Example:

Use a Normal distribution with a mean value of 5 and a standard deviation of 1 to generate 100 random numbers.

 

 

 

 

 

 

 

Create a line chart of the sample data.

 

From the Context Panel for the generated data, select Statistics>Visualization>Histogram.

 

After previewing the graph, click Quit to return the Histogram to your document.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

withStatistics:

XRandomVariableNormal5,1

X_R

(3.5)

SampleX,100

1 .. 100 VectorrowData Type: float8Storage: rectangularOrder: Fortran_order

(3.6)

histogram

 

You can also define a random number generator using a distribution.

 

Here, X1 is a procedure that takes n as an argument, where n is the number of random numbers to generate.

 

Each time you use X1, it generates new random numbers.

 

 

 

 

 

 

 

 

Example:

Generate 10 random numbers using X1(10) and then create a line chart of the data using the LineChart command.

X1SampleRandomVariableNormal5,1

X1procn::sizeType...end proc

(3.7)

X11

5.56775664743282

(3.8)

X11

4.94414995600839

(3.9)

X13

5.614959832078395.299582649024375.30232746471033

(3.10)

DataX110

Data4.632833451091855.415785504934513.657682545931623.236606024517473.922223312544215.415879165099574.577452943142294.915882479005063.890643799254303.94021634676111

(3.11)

LineChartData,color=red,thickness=3,symbol=circle,symbolsize=10,legend=Data

 

Statistics, Regression, and Curve Fitting

Basic Tools

Steps

Results

With Maple, it is easy to use statistics and to do curve fitting and regression analysis.

 

Many useful commands are available directly from the Context Panel.

 

Easy Access to Statistics Operations

 

For a 1-dimensional array, vector or list, the Context Panel includes Statistics operations.  Under Statistics, the categories Data Manipulation, Quantities, Summary and Tabulation, and Visualization offer many Statistics commands.

 

Example:

Define a list as shown.  In the Context Panel for the output, choose Statistics>Quantities>Mean.

seqln2.3 i,i=1..12

0.8329091229,1.526056303,1.931521412,2.219203484,2.442347035,2.624668592,2.778819272,2.912350665,3.030133700,3.135494216,3.230804396,3.317815773

(4.1.1)

mean

2.49851033090833

(4.1.2)

The Curve Fitting Assistant is also available through the Context Panel.

 

Example:  

In this example, we start by listing the pairs of data.

 

In the Context Panel for the data, select Curve Fitting>Interactive Curve Fitting. Select OK when prompted to specify the variable name.  This opens the Curve Fitting Assistant which provides an easy way to access Maple's curve fitting commands.  

 

You can choose the type of curve and preview the result.  The function is displayed below the plot.  Select the Splines curve, and select the corresponding Plot button.

 

You can opt to return the function (interpolant) or the plot. In the drop-down menu near the bottom left corner of the window, change the selection from Interpolant to Plot. Click Done.

 

The plot is displayed.

 

 

 

0,3,1,1,2,7,3,6,4,9, 5,11

0,3,1,1,2,7,3,6,4,9,5,11

(4.1.3)

curve fitting assistant

You can also call the Curve Fitting Assistant with a command.  This allows you to pass data that was defined earlier in the Maple session.

 

Example:  Use the DataSet that was imported in the Importing Data section of this tutorial.  (If you have not done so already, go to this section first and follow the steps to import and define the data.)

 

1. 

Enter the command "CurveFitting[Interactive](DataSet)".  The Curve Fitting Assistant opens.

 

2. 

Under Least Squares, click Plot to see the least squares curve.  

 

3. 

At the bottom of the window, set the option to return the plot.  In the drop-down menu near the bottom left corner of the window, change the selection from Interpolant to Plot.  Click Done.

 

The plot is displayed.

 

CurveFittingInteractiveDataSet;

Advanced Example: Noisy Signal

Steps

Results

In this example, you will use a random number generator to add noise to a signal.

 

Define the noise as a random number generator with mean value 1 and standard deviation 0.5.


We use a sample size of 200 to create a vector of noise values.

 

We can then extract entries from the vector.  The vector indices go from 1 to 200.  To select the ith entry, use the selection operator: PureNoiseVector[i].

withStatistics:

NoiseSampleRandomVariableNormal1,0.5:

PureNoiseVectorNoise200:

PureNoiseVector1

1.14175247336227

(4.2.1)

Define a signal.

 

Create a noisy signal by adding noise to the original signal:

signal0.1i100+PureNoiseVector[i]

 

 

Generate a data set using the seq command.

 

 

 

 

Plot the signal and noisyData together.

signal:=x→xsinx:

noisyDataseq0.1i100,signal0.1i100+PureNoiseVectori,i=1..200:

plotnoisyData,signalx,style=point,symbol=cross,circle,color=Green,Blue,legend=Noisy Signal,Signal

Curve Fitting

 

To fit a functional model to the noisy data, use the Fit command from the Statistics package.  

 

The calling sequence is Fit(f, X, Y, v), where f is the function model, X and Y are the x and y data, and v is the name of the independent variable in the component function.

 

Create two lists Xdata and Ydata from the data in the two columns of noisyData.

 

Tip: It's often useful to do something for each element of a data structure.  Here, we do this by i=1..nops(noisyData).  Recall, nops is the number of elements.

 

The function model is f=cxsinax+b.

 

 

 

Plot f and the noisyData together to visualize the fit.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Tip:  The Statistics and CurveFitting packages contain a number of commands for fitting curves to data points including linear, exponential, polynomial, least squares, and spline.

XdataseqnoisyDatai,1,i=1..nopsnoisyData:

YdataseqnoisyDatai,2,i=1..nopsnoisyData:

fFitcxsinax+b,Xdata,Ydata,x

f1.01118654034700xsin1.00093056049853x+1.08883811779035

(4.2.2)

plotnoisyData,fx,x=10..10,style=point,line,symbol=cross,legend=Noisy Signal, Fitted Model

Exercise:  Repeat the noisy signal example using a random number generator based on a Weibull distribution.

 

See Also

Array, cat, currentdir, Curve Fitting (Maple Portal topic), CurveFitting package, kernelopts(datadir), list, map, rand, seq, Statistics

 

Go to Maple Portal     Previous Tutorial     Next Tutorial