-
Notifications
You must be signed in to change notification settings - Fork 2
Expand file tree
/
Copy pathfileio.html
More file actions
347 lines (319 loc) · 23.7 KB
/
fileio.html
File metadata and controls
347 lines (319 loc) · 23.7 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta charset="utf-8" />
<title>Reading HSI Data Files — Spectral Python 0.21 documentation</title>
<link rel="stylesheet" href="_static/classic.css" type="text/css" />
<link rel="stylesheet" href="_static/pygments.css" type="text/css" />
<script id="documentation_options" data-url_root="./" src="_static/documentation_options.js"></script>
<script src="_static/jquery.js"></script>
<script src="_static/underscore.js"></script>
<script src="_static/doctools.js"></script>
<script src="_static/language_data.js"></script>
<link rel="search" title="Search" href="search.html" />
<link rel="next" title="Displaying Data" href="graphics.html" />
<link rel="prev" title="Installing SPy" href="installation.html" />
</head><body>
<div class="related" role="navigation" aria-label="related navigation">
<h3>Navigation</h3>
<ul>
<li class="right" style="margin-right: 10px">
<a href="py-modindex.html" title="Python Module Index"
>modules</a></li>
<li class="right" >
<a href="graphics.html" title="Displaying Data"
accesskey="N">next</a> |</li>
<li class="right" >
<a href="installation.html" title="Installing SPy"
accesskey="P">previous</a> |</li>
<li class="nav-item nav-item-0"><a href="index.html">Spectral Python 0.21 documentation</a> »</li>
<li class="nav-item nav-item-1"><a href="user_guide.html" accesskey="U">Spectral Python (SPy) User Guide</a> »</li>
</ul>
</div>
<div class="document">
<div class="documentwrapper">
<div class="bodywrapper">
<div class="body" role="main">
<div class="toctree-wrapper compound" id="fileio">
</div>
<div class="section" id="reading-hsi-data-files">
<span id="reading-data-files"></span><h1>Reading HSI Data Files<a class="headerlink" href="#reading-hsi-data-files" title="Permalink to this headline">¶</a></h1>
<p>The standard means of opening and accessing a hyperspectral image file with SPy
is via the <code class="xref py py-func docutils literal notranslate"><span class="pre">image</span></code> function, which returns an instance of a
<a class="reference internal" href="class_func_ref.html#spectral.SpyFile" title="spectral.SpyFile"><code class="xref py py-class docutils literal notranslate"><span class="pre">SpyFile</span></code></a> object.</p>
<div class="section" id="module-spectral.io.spyfile">
<span id="the-spyfile-interface"></span><span id="spyfile-interface"></span><h2>The SpyFile Interface<a class="headerlink" href="#module-spectral.io.spyfile" title="Permalink to this headline">¶</a></h2>
<p><a class="reference internal" href="class_func_ref.html#spectral.SpyFile" title="spectral.SpyFile"><code class="xref py py-class docutils literal notranslate"><span class="pre">SpyFile</span></code></a> is the base class for creating objects to read
hyperspectral data files. When a <a class="reference internal" href="class_func_ref.html#spectral.SpyFile" title="spectral.SpyFile"><code class="xref py py-class docutils literal notranslate"><span class="pre">SpyFile</span></code></a> object is created,
it provides an interface to read data from a corresponding file. When an image
is opened, the actual object returned will be a subclass of
<a class="reference internal" href="class_func_ref.html#spectral.SpyFile" title="spectral.SpyFile"><code class="xref py py-class docutils literal notranslate"><span class="pre">SpyFile</span></code></a> (BipFile, BilFile, or BsqFile) corresponding to the
interleave of the data within the image file.</p>
<p>Let’s open our sample image.</p>
<pre class="literal-block">In [1]: from spectral import *
In [2]: img = open_image('92AV3C.lan')
In [3]: img.__class__
Out[3]: spectral.io.bilfile.BilFile
In [4]: print(img)
Data Source: '/home/thomas/spectral_data/92AV3C.lan'
# Rows: 145
# Samples: 145
# Bands: 220
Interleave: BIL
Quantization: 16 bits
Data format: int16</pre>
<p>The image was not located in the working directory but it was still opened
because it was in a directory specified by the <em>SPECTRAL_DATA</em> environment
variable. Because the image pixel data are interleaved by line, the <em>image</em>
function returned a <em>BilFile</em> instance.</p>
<p>Since hyperspectral image files can be quite large, only
metadata are read from the file when the <a class="reference internal" href="class_func_ref.html#spectral.SpyFile" title="spectral.SpyFile"><code class="xref py py-class docutils literal notranslate"><span class="pre">SpyFile</span></code></a> object is
first created. Image data values are only read when specifically requested via
<a class="reference internal" href="class_func_ref.html#spectral.SpyFile" title="spectral.SpyFile"><code class="xref py py-class docutils literal notranslate"><span class="pre">SpyFile</span></code></a> methods. The <a class="reference internal" href="class_func_ref.html#spectral.SpyFile" title="spectral.SpyFile"><code class="xref py py-class docutils literal notranslate"><span class="pre">SpyFile</span></code></a> class
provides a subscript operator that behaves much like the numpy array subscript
operator. The <a class="reference internal" href="class_func_ref.html#spectral.SpyFile" title="spectral.SpyFile"><code class="xref py py-class docutils literal notranslate"><span class="pre">SpyFile</span></code></a> object is subscripted as an <em>MxNxB</em>
array where <em>M</em> is the number of rows in the image, <em>N</em> is the number of
columns, and <em>B</em> is thenumber of bands.</p>
<div class="highlight-ipython notranslate"><div class="highlight"><pre><span></span><span class="gp">In [5]: </span><span class="n">img</span><span class="o">.</span><span class="n">shape</span>
<span class="gh">Out[5]: </span><span class="go">(145, 145, 220)</span>
<span class="gp">In [6]: </span><span class="n">pixel</span> <span class="o">=</span> <span class="n">img</span><span class="p">[</span><span class="mi">50</span><span class="p">,</span><span class="mi">100</span><span class="p">]</span>
<span class="gp">In [7]: </span><span class="n">pixel</span><span class="o">.</span><span class="n">shape</span>
<span class="gh">Out[7]: </span><span class="go">(220,)</span>
<span class="gp">In [8]: </span><span class="n">band6</span> <span class="o">=</span> <span class="n">img</span><span class="p">[:,:,</span><span class="mi">5</span><span class="p">]</span>
<span class="gp">In [9]: </span><span class="n">band6</span><span class="o">.</span><span class="n">shape</span>
<span class="gh">Out[9]: </span><span class="go">(145, 145, 1)</span>
</pre></div>
</div>
<p>The image data values were not read from the file until the subscript operator
calls were performed. Note that since Python indices start at 0,
<code class="docutils literal notranslate"><span class="pre">img[50,100]</span></code> refers to the pixel at 51st row and 101st column of the image.
Similarly, <code class="docutils literal notranslate"><span class="pre">img[:,:,5]</span></code> refers to all the rows and columns for the 6th band
of the image.</p>
<p><a class="reference internal" href="class_func_ref.html#spectral.SpyFile" title="spectral.SpyFile"><code class="xref py py-class docutils literal notranslate"><span class="pre">SpyFile</span></code></a> subclass instances returned for particular image
files will also provide the following methods:</p>
<table class="docutils align-default">
<colgroup>
<col style="width: 18%" />
<col style="width: 82%" />
</colgroup>
<thead>
<tr class="row-odd"><th class="head"><p>Method</p></th>
<th class="head"><p>Description</p></th>
</tr>
</thead>
<tbody>
<tr class="row-even"><td><p>read_band</p></td>
<td><p>Reads a single band into an <em>MxN</em> array</p></td>
</tr>
<tr class="row-odd"><td><p>read_bands</p></td>
<td><p>Reads multiple bands into an <em>MxNxC</em> array</p></td>
</tr>
<tr class="row-even"><td><p>read_pixel</p></td>
<td><p>Reads a single pixel into a length <em>B</em> array</p></td>
</tr>
<tr class="row-odd"><td><p>read_subregion</p></td>
<td><p>Reads multiple bands from a rectangular sub-region of the image</p></td>
</tr>
<tr class="row-even"><td><p>read_subimage</p></td>
<td><p>Reads specified rows, columns, and bands</p></td>
</tr>
</tbody>
</table>
<p><a class="reference internal" href="class_func_ref.html#spectral.SpyFile" title="spectral.SpyFile"><code class="xref py py-class docutils literal notranslate"><span class="pre">SpyFile</span></code></a> objects have a <code class="docutils literal notranslate"><span class="pre">bands</span></code> member, which is an
instance of a <a class="reference internal" href="class_func_ref.html#spectral.BandInfo" title="spectral.BandInfo"><code class="xref py py-class docutils literal notranslate"><span class="pre">BandInfo</span></code></a> object that contains optional
information about the images spectral bands.</p>
</div>
<div class="section" id="loading-entire-images">
<span id="loading-images"></span><h2>Loading Entire Images<a class="headerlink" href="#loading-entire-images" title="Permalink to this headline">¶</a></h2>
<p>It is important to note that image data are read by a <a class="reference internal" href="class_func_ref.html#spectral.SpyFile" title="spectral.SpyFile"><code class="xref py py-class docutils literal notranslate"><span class="pre">SpyFile</span></code></a> object on demand
and the data are not cached. Each time the <a class="reference internal" href="class_func_ref.html#spectral.SpyFile" title="spectral.SpyFile"><code class="xref py py-class docutils literal notranslate"><span class="pre">SpyFile</span></code></a> subscript operator or one of
the <a class="reference internal" href="class_func_ref.html#spectral.SpyFile" title="spectral.SpyFile"><code class="xref py py-class docutils literal notranslate"><span class="pre">SpyFile</span></code></a> read methods are called, data are read from the corresponding image
data file, regardless of whether the same data have been previously read. This
is done to avoid consuming too much memory when working with very large image files.
It also improves performance when performing operations that only require reading
a small portion of the data in a large image (e.g., reading RGB bands to display
the image). The downside of reading data on demand and not caching the data is that there can
be a significant run time penalty when running algorithms that require access to
all of the data. Performance will be even worse if the algorithm requires iterative
access to the data.</p>
<p>To improve performance of spectral algorithms, it is preferable to load the entire
image into memory using the <a class="reference internal" href="class_func_ref.html#spectral.SpyFile.load" title="spectral.SpyFile.load"><code class="xref py py-meth docutils literal notranslate"><span class="pre">load</span></code></a> method, which returns
an <code class="xref py py-class docutils literal notranslate"><span class="pre">ImageArray</span></code> object. <code class="xref py py-class docutils literal notranslate"><span class="pre">ImageArray</span></code> provides
the full <code class="xref py py-class docutils literal notranslate"><span class="pre">numpy.ndarray</span></code> interface, as well as the <a class="reference internal" href="class_func_ref.html#spectral.SpyFile" title="spectral.SpyFile"><code class="xref py py-class docutils literal notranslate"><span class="pre">SpyFile</span></code></a> interface.</p>
<pre class="literal-block">In [1]: arr = img.load()
In [2]: arr.__class__
Out[2]: spectral.image.ImageArray
In [3]: print(arr.info())
# Rows: 145
# Samples: 145
# Bands: 220
Data format: float32
In [4]: arr.shape
Out[4]: (145, 145, 220)</pre>
<p>Because SPy is primarily designed for processing in the spectral domain,
<code class="xref py py-class docutils literal notranslate"><span class="pre">spectral.ImageArray</span></code> objects in memory will always have data interleaved
by pixel, regardless of the interleave of the source image data file. In other
words, the <code class="xref py py-class docutils literal notranslate"><span class="pre">numpy.ndarray</span></code> shape will be <code class="docutils literal notranslate"><span class="pre">(numRows,</span> <span class="pre">numCols,</span> <span class="pre">numBands)</span></code>.
<code class="xref py py-class docutils literal notranslate"><span class="pre">ImageArray</span></code> objects always contain 32-bit floats.</p>
<div class="admonition note">
<p class="admonition-title">Note</p>
<p>Before calling the <code class="xref py py-meth docutils literal notranslate"><span class="pre">load</span></code> method, it is important to consider the amount of memory
that will be consumed by the resulting ImageArray object. Since <code class="xref py py-class docutils literal notranslate"><span class="pre">spectral.ImageArray</span></code> uses
32-bit floating point values, the amount of memory consumed will be approximately
<code class="docutils literal notranslate"><span class="pre">4</span> <span class="pre">*</span> <span class="pre">numRows</span> <span class="pre">*</span> <span class="pre">numCols</span> <span class="pre">*</span> <span class="pre">numBands</span></code> bytes.</p>
</div>
</div>
<div class="section" id="numpy-memmap-interface">
<h2>NumPy <code class="xref py py-class docutils literal notranslate"><span class="pre">memmap</span></code> Interface<a class="headerlink" href="#numpy-memmap-interface" title="Permalink to this headline">¶</a></h2>
<p>As an alternative to loading an entire image into memory, a somewhat slower
(but more memory efficient) way to access image data is to use a numpy memmap
object, as returned by the <a class="reference internal" href="class_func_ref.html#spectral.io.bipfile.BipFile.open_memmap" title="spectral.io.bipfile.BipFile.open_memmap"><code class="xref py py-meth docutils literal notranslate"><span class="pre">open_memmap</span></code></a>
method of SpyFile objects. memmap objects can also be used to write date to
an image file.</p>
</div>
<div class="section" id="file-formats-supported">
<span id="file-formats"></span><h2>File Formats Supported<a class="headerlink" href="#file-formats-supported" title="Permalink to this headline">¶</a></h2>
<div class="section" id="envi-headers">
<span id="envi-format"></span><h3>ENVI Headers<a class="headerlink" href="#envi-headers" title="Permalink to this headline">¶</a></h3>
<p>ENVI <a class="footnote-reference brackets" href="#envi-trademark" id="id1">1</a> is a popular commercial software package for processing and analyzing
geospatial imagery. SPy can read images that have associated ENVI header files
and can read & write spectral libraries with ENVI headers. ENVI files are opened
automatically by the SPy <code class="xref py py-func docutils literal notranslate"><span class="pre">image</span></code> function but images can also be
opened explicitly as ENVI files. It may be necessary to open an ENVI file explicitly
if the data file is in a separate directory from the header or if the data file
has an unusual file extension that SPy can not identify.</p>
<div class="highlight-ipython notranslate"><div class="highlight"><pre><span></span><span class="gp">In [5]: </span><span class="kn">import</span> <span class="nn">spectral.io.envi</span> <span class="k">as</span> <span class="nn">envi</span>
<span class="gp">In [6]: </span><span class="n">img</span> <span class="o">=</span> <span class="n">envi</span><span class="o">.</span><span class="n">open</span><span class="p">(</span><span class="s1">'cup95eff.int.hdr'</span><span class="p">,</span> <span class="s1">'cup95eff.int'</span><span class="p">)</span>
<span class="gp">In [7]: </span><span class="kn">import</span> <span class="nn">spectral.io.envi</span> <span class="k">as</span> <span class="nn">envi</span>
<span class="gp">In [8]: </span><span class="n">lib</span> <span class="o">=</span> <span class="n">envi</span><span class="o">.</span><span class="n">open</span><span class="p">(</span><span class="s1">'spectra.hdr'</span><span class="p">)</span>
<span class="gp">In [9]: </span><span class="n">lib</span><span class="o">.</span><span class="n">names</span><span class="p">[:</span><span class="mi">5</span><span class="p">]</span>
<span class="gh">Out[9]: </span><span class="go"></span>
<span class="go">['construction asphalt',</span>
<span class="go"> 'construction concrete',</span>
<span class="go"> 'red smooth-faced brick',</span>
<span class="go"> 'weathered red brick',</span>
<span class="go"> 'bare red brick']</span>
</pre></div>
</div>
<div class="admonition seealso">
<p class="admonition-title">See also</p>
<p>Functions for writing image data to files:</p>
<p><a class="reference internal" href="class_func_ref.html#spectral.io.envi.create_image" title="spectral.io.envi.create_image"><code class="xref py py-func docutils literal notranslate"><span class="pre">create_image</span></code></a>:</p>
<blockquote>
<div><p>Creates a new image file with allocated storage on disk.</p>
</div></blockquote>
<p><a class="reference internal" href="class_func_ref.html#spectral.io.envi.save_image" title="spectral.io.envi.save_image"><code class="xref py py-func docutils literal notranslate"><span class="pre">save_image</span></code></a>:</p>
<blockquote>
<div><p>Saves an existing image or ndarray to a file with an ENVI header.</p>
</div></blockquote>
</div>
</div>
<div class="section" id="aviris">
<h3>AVIRIS<a class="headerlink" href="#aviris" title="Permalink to this headline">¶</a></h3>
<p>SPy supports data files generated by the Airborne Visible/Infrared Imaging
Spectrometer (AVIRIS) <a class="footnote-reference brackets" href="#id3" id="id2">2</a>. AVIRIS files are automatically recognized by
the <code class="xref py py-func docutils literal notranslate"><span class="pre">open_image</span></code> function; however, spectral band calibration files
are not automatically recognized; therefore you may want to open the image as
an AVIRIS file explicitly and specify the cal file.</p>
<div class="highlight-ipython notranslate"><div class="highlight"><pre><span></span><span class="gp">In [10]: </span><span class="n">img</span> <span class="o">=</span> <span class="n">aviris</span><span class="o">.</span><span class="n">open</span><span class="p">(</span><span class="s1">'f970619t01p02_r02_sc01.a.rfl'</span><span class="p">,</span> <span class="s1">'f970619t01p02_r02.a.spc'</span><span class="p">)</span>
</pre></div>
</div>
<p>You can also load the band calibration file separately (this may be necessary if
the band calibration file is in AVIRIS format but the image is not).</p>
<div class="highlight-ipython notranslate"><div class="highlight"><pre><span></span><span class="gp">In [11]: </span><span class="n">img</span> <span class="o">=</span> <span class="n">open_image</span><span class="p">(</span><span class="s1">'92AV3C.lan'</span><span class="p">)</span>
<span class="gp">In [12]: </span><span class="n">img</span><span class="o">.</span><span class="n">bands</span> <span class="o">=</span> <span class="n">aviris</span><span class="o">.</span><span class="n">read_aviris_bands</span><span class="p">(</span><span class="s1">'92AV3C.spc'</span><span class="p">)</span>
</pre></div>
</div>
</div>
<div class="section" id="erdas-lan">
<h3>ERDAS/Lan<a class="headerlink" href="#erdas-lan" title="Permalink to this headline">¶</a></h3>
<p>The ERDAS/Lan file format is automatically recognized by <code class="xref py py-func docutils literal notranslate"><span class="pre">image</span></code>.
It is unlikely that a file would need to be opened explicitly as a Lan file but it
can be done as follows.</p>
<div class="highlight-ipython notranslate"><div class="highlight"><pre><span></span><span class="gp">In [13]: </span><span class="kn">import</span> <span class="nn">spectral.io.erdas</span> <span class="k">as</span> <span class="nn">erdas</span>
<span class="gp">In [14]: </span><span class="n">img</span> <span class="o">=</span> <span class="n">erdas</span><span class="o">.</span><span class="n">open</span><span class="p">(</span><span class="s1">'92AV3C.lan'</span><span class="p">)</span>
</pre></div>
</div>
<dl class="footnote brackets">
<dt class="label" id="envi-trademark"><span class="brackets"><a class="fn-backref" href="#id1">1</a></span></dt>
<dd><p>ENVI is a registered trademark of Exelis Visual Information Solutions.</p>
</dd>
<dt class="label" id="id3"><span class="brackets"><a class="fn-backref" href="#id2">2</a></span></dt>
<dd><p><a class="reference external" href="http://aviris.jpl.nasa.gov/">http://aviris.jpl.nasa.gov/</a></p>
</dd>
</dl>
</div>
</div>
</div>
</div>
</div>
</div>
<div class="sphinxsidebar" role="navigation" aria-label="main navigation">
<div class="sphinxsidebarwrapper">
<h3><a href="index.html">Table of Contents</a></h3>
<ul>
<li><a class="reference internal" href="#">Reading HSI Data Files</a><ul>
<li><a class="reference internal" href="#module-spectral.io.spyfile">The SpyFile Interface</a></li>
<li><a class="reference internal" href="#loading-entire-images">Loading Entire Images</a></li>
<li><a class="reference internal" href="#numpy-memmap-interface">NumPy <code class="xref py py-class docutils literal notranslate"><span class="pre">memmap</span></code> Interface</a></li>
<li><a class="reference internal" href="#file-formats-supported">File Formats Supported</a><ul>
<li><a class="reference internal" href="#envi-headers">ENVI Headers</a></li>
<li><a class="reference internal" href="#aviris">AVIRIS</a></li>
<li><a class="reference internal" href="#erdas-lan">ERDAS/Lan</a></li>
</ul>
</li>
</ul>
</li>
</ul>
<h4>Previous topic</h4>
<p class="topless"><a href="installation.html"
title="previous chapter">Installing SPy</a></p>
<h4>Next topic</h4>
<p class="topless"><a href="graphics.html"
title="next chapter">Displaying Data</a></p>
<div id="searchbox" style="display: none" role="search">
<h3 id="searchlabel">Quick search</h3>
<div class="searchformwrapper">
<form class="search" action="search.html" method="get">
<input type="text" name="q" aria-labelledby="searchlabel" />
<input type="submit" value="Go" />
</form>
</div>
</div>
<script>$('#searchbox').show(0);</script>
</div>
</div>
<div class="clearer"></div>
</div>
<div class="related" role="navigation" aria-label="related navigation">
<h3>Navigation</h3>
<ul>
<li class="right" style="margin-right: 10px">
<a href="py-modindex.html" title="Python Module Index"
>modules</a></li>
<li class="right" >
<a href="graphics.html" title="Displaying Data"
>next</a> |</li>
<li class="right" >
<a href="installation.html" title="Installing SPy"
>previous</a> |</li>
<li class="nav-item nav-item-0"><a href="index.html">Spectral Python 0.21 documentation</a> »</li>
<li class="nav-item nav-item-1"><a href="user_guide.html" >Spectral Python (SPy) User Guide</a> »</li>
</ul>
</div>
<div class="footer" role="contentinfo">
© Copyright 2014, Thomas Boggs.
Created using <a href="http://sphinx-doc.org/">Sphinx</a> 2.4.4.
</div>
<!-- Google Analytics -->
<script>
(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
})(window,document,'script','//www.google-analytics.com/analytics.js','ga');
ga('create', 'UA-47389515-2', 'spectralpython.net');
ga('send', 'pageview');
</script>
<!-- End Google Analytics -->
</body>
</html>