Updates paths and reorganize.

[nguy]

Updated radar loop for current to paths for pyart/vispy packages.
Reorganized the example a bit to begin to generalize.
Trailing whitespace removed by my editor.

[nguy]

This PR should NOT be merged until we work through issues.

When I run this, the dummy data appears, but there is not image update.
The time structure appears to be picky and I've attempted to get to the bottom of this, without a lot of luck.

I receive the following error message when running:

guy:vispy $ python vispy_radar_loop_demo.py 
WARNING: OpenGL version 3.0 or higher recommended, got 2.1 NVIDIA-10.4.2 310.41.35f01. Some functionality may fail.
2014-06-07T21:10:00-0600
WARNING: Traceback (most recent call last):
  File "vispy_radar_loop_demo.py", line 303, in <module>
    vispy.app.run()
  File "/Users/guy/anaconda/lib/python2.7/site-packages/vispy/app/_default_app.py", line 61, in run
    return default_app.run()
  File "/Users/guy/anaconda/lib/python2.7/site-packages/vispy/app/application.py", line 123, in run
    return self._backend._vispy_run()
  File "/Users/guy/anaconda/lib/python2.7/site-packages/vispy/app/backends/_qt.py", line 217, in _vispy_run
    return app.exec_()
  File "/Users/guy/anaconda/lib/python2.7/site-packages/vispy/app/backends/_qt.py", line 661, in _vispy_timeout
    self._vispy_timer._timeout()
  File "/Users/guy/anaconda/lib/python2.7/site-packages/vispy/app/timer.py", line 168, in _timeout
    count=self.iter_count)
  File "/Users/guy/anaconda/lib/python2.7/site-packages/vispy/util/event.py", line 436, in __call__
    self._invoke_callback(cb, event)
  File "/Users/guy/anaconda/lib/python2.7/site-packages/vispy/util/event.py", line 452, in _invoke_callback
    self, cb_event=(cb, event))
  << caught exception here: >>
  File "/Users/guy/anaconda/lib/python2.7/site-packages/vispy/util/event.py", line 448, in _invoke_callback
    cb(event)
  File "vispy_radar_loop_demo.py", line 196, in loop_radar
    fieldnames=(self.radar_fields[0],))
  File "/Users/guy/software/tmp/vispy/collection.py", line 142, in sweep_data_for_time_range
    filename, ray_slice = self.sweep_for_time_range(t0, t1, **kwargs)
  File "/Users/guy/software/tmp/vispy/collection.py", line 124, in sweep_for_time_range
    overlap = ((t0 - self.sweep_table['end'].values) < 0) & ((t1 - self.sweep_table['start'].values) > 0 )
TypeError: Cannot cast ufunc less input from dtype('<m8[ns]') to dtype('<m8') with casting rule 'same_kind'
ERROR: Invoking <bound method Canvas.loop_radar of <Canvas (PyQt4) at 0x10f817d50>> for Event
2014-06-07T21:10:10-0600
ERROR: Invoking <bound method Canvas.loop_radar of <Canvas (PyQt4) at 0x10f817d50>> repeat 2
2014-06-07T21:10:20-0600
2014-06-07T21:10:30-0600
ERROR: Invoking <bound method Canvas.loop_radar of <Canvas (PyQt4) at 0x10f817d50>> repeat 4

[deeplycloudy]

I suspect the problem is with (t0 - self.sweep_table['end'].values), due to a difference in the units attached to the datetime dtypes associated with t0 and the time data in the sweep_table. One of them has units of nanoseconds, while the other seems to have no units.

According to these docs, numpy will indeed complain about mismatches. http://docs.scipy.org/doc/numpy-1.10.0/reference/arrays.datetime.html

On numpy 1.9.3 I don't have this problem.

Can you send your numpy version, and also print the dtypes of t0, t1, self.sweep_table['end'].values, and self.sweep_table['start'].values?

If those all match, then I bet it's a problem with the cast from 0 to a datetime type.

The solution is probably to give numpy a hint as to how to make the conversion so no automatic cast is necessary.

[nguy]

I'm running numpy 1.10.1

Data types as is:

t0/t1 type = datetime64[s]/datetime64[s]
sweep_table['end'] type = datetime64[ns]
sweep_table['start'] type = datetime64[ns]

If I change the code to input start and stop in ns:

t0/t1 type = datetime64[ns]/datetime64[ns]
sweep_table['end'] type = datetime64[ns]
sweep_table['start'] type = datetime64[ns]

No change in error:
TypeError: Cannot cast ufunc less input from dtype('<m8[ns]') to dtype('<m8') with casting rule 'same_kind'

[deeplycloudy]

After updating numpy, I still didn't get the error. Maybe it's an OS thing.

I just noticed that the ufunc is "less," which means it is the less than comparison. The following code should be equivalent. Does it fix your error?

dt_zero = np.timedelta64(0,'ns')
overlap = ((t0 - self.sweep_table['end']) < dt_zero ) & ((t1 - self.sweep_table['start']) > dt_zero )

Incidentally, I also had to change line 105 of quadmesh_geometry to fix a type casting issue after upgrading numpy. I guess the type checking is more strict.
adjust_start = (np.arange(M-1, dtype='i4')[:,None]*np.ones(N-1, dtype='i4')[None,:]).flatten()

[nguy]

That worked, though I didn't seem to need to change the second line you noted.
Nice catch on the zero, that is some very strict type checking!

Next issue :) :
My display is not updating properly. I don't think the mapping is correct and maybe this is due to the fact that I changed the transform from MatrixTransform to STTransform. I've pushed the latest code changes for reference.

Furthermore, when I rotate the image to horizontal I see a mapping of data, just not lining up with the dummy sweep coordinates.

[deeplycloudy]

Is it possible you're on an older vispy? I just updated to vispy/master and everything still works fine for me. They've moved those imports around quite a bit.

The transform/scaling process to get the radar in the right place is really sensitive, and I've lost radars this way before …

[nguy]

What version of vispy are you using? Did you install from source?

Are you testing on the code I submitted with the change away from MatrixTransform?

[deeplycloudy]

I installed from source (git master). I haven't pulled your fork to test yet, just testing on my old, unmodified example.

[nguy]

So MatrixTransform works for you then? Sorry trying to figure out what's happening here.

[deeplycloudy]

Yeah, matrix transform, as in the original brawl4d, works fine for me. I don't see how those imports would fail if we're on the same vispy. In my initial email correspondence, I had assumed you were on a newer vispy than me, but I suspect you might now be on an older version?

[nguy]

Wow, yep I was using a conda distribution of vispy that was older (v0.4.106) instead of the 0.5.0 developement version. Pretty substantial differences it appears!

[nguy]

So now it seems to be working. I'll probably make a few more changes to the class structure and resubmit the PR for you to look at. After that I'll try to build a plugin from ARTView to just try to use the existing method and we can talk about advancing that later.

I'll respond to your email so we can hash out a initial roadmap.

Thanks for the help!

[deeplycloudy]

Ah, great to hear! Vispy is very much in alpha, still trying to figure out the best abstractions and how to reflect that in the module hierarchy, so I've made a point to keep tracking their master.

Look forward to hearing your thoughts on a roadmap.

[deeplycloudy]

@nguy Do the Ka data appear when you run your version of the demo? The 88D data seem to update just fine when I run your branch, but I don't see the Ka slice.

[nguy]

No it doesn't. I was so happy that it was working I forgot there were two data sets.
Hmm.

[deeplycloudy]

I see you've refactored for an arbitrary number of radar file collections, which I like. It probably got lost in that refactoring step. I'll see if I can figure out what's going on.

[deeplycloudy]

The meshes for each radar file collection weren't being looped over, plus there was some elevation filtering logic that I originally implemented for the 88D data that was preventing the Ka RHIs from displaying. I fixed that. It would be desirable to have a general-purpose version of this class be able to subset by azimuth or range on a per-radar basis, but that requires some design discussion that is not part of this PR.

I also changed to the docstring to indicate that even a single radar's files should be specified as a list of lists, since it seemed like the code required that. I think a single call signature is preferable to having to maintain a bunch of special casing in the code, too.

There is probably further streamlining to be done in the code, but I wanted to verify this works for you before doing cleanup.

Because I don't entirely know what I'm doing on github, I pushed my additional changes to the nguy-master branch, instead of updating your previous pull request (if that's even possible). If I need to do something else, I'd welcome instruction.