[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [geomesa-users] Need help on mismatch of returned data

Title: Default Disclaimer Daimler AG
Hello,

I notice that in your code, you are using an 'equals' query on VIN, while on the command-line you are using a 'like' with wildcards. Could you try using a consistent predicate between the two? I'm not entirely sure that 'like' is handled correctly in the FSDS, tbh.

Thanks,

Emilio

On 5/10/19 3:59 AM, christian.sickert@xxxxxxxxxxx wrote:

Hi Emilio,

 

This is Christian, a colleague of Markus - the initial requester, who is currently out-of-office. Maybe I can answer your question.

 

We’re using the GeoTools API to query the data. We basically do it in the same way as described in the geomesa-tutorial https://github.com/geomesa/geomesa-tutorials/blob/master/geomesa-tutorials-common/src/main/java/org/geomesa/example/transformations/GeoMesaQueryTutorial.java

 

First, we build an org.geotools.data.Query instance using org.opengis.filter.FilterFactory2. We’re applying temporal and spatial constraints as well as a constraint on a custom attribute called VIN:

 

Query query = new Query(typeName);
List<Filter> filters =
new ArrayList<>();

FilterFactory2 filterFactory = CommonFactoryFinder.getFilterFactory2();

// temporal filter:
filters.add(filterFactory.between(
      filterFactory.property(EventSimpleFeature.
DATE_TIME_GROUP),
      filterFactory.literal(
dateFrom),
      filterFactory.literal(
dateTo)));

// spatial filter:
filters.add(filterFactory
      .bbox(filterFactory.property(EventSimpleFeature.
GEOMETRY),
           
boundingBox.getMinX(), boundingBox.getMinY(),
           
boundingBox.getMaxX(), boundingBox.getMaxY(), "EPSG:4326"));

// vin filter:
filters.add(filterFactory
      .equals(filterFactory.property(EventSimpleFeature.
VIN), filterFactory.literal(vin)));

query.setFilter(filterFactory.and(filters));
query.setPropertyNames(
fieldNames);
return query;

 

With that query we then instantiate an org.geotools.data.FeatureReader to read the filtered (according the given query) entries:

 

FeatureReader<SimpleFeatureType, SimpleFeature> reader = dataStore.getFeatureReader(query, Transaction.AUTO_COMMIT);
while (reader.hasNext()) {
   SimpleFeature feature = reader.next();

   // do something with the feature ...
}

 

Is that sufficient for a further analysis on your side?

 

Thanks for your support!

 

Best regards

Christian

 

 

Von: geomesa-users-bounces@xxxxxxxxxxxxxxxx <geomesa-users-bounces@xxxxxxxxxxxxxxxx> Im Auftrag von Emilio Lahr-Vivaz
Gesendet: Mittwoch, 8. Mai 2019 22:32
An: geomesa-users@xxxxxxxxxxxxxxxx
Betreff: Re: [geomesa-users] Need help on mismatch of returned data

 

Hello,

Could you provide the code you're using to query programmatically?

Thanks,

Emilio

On 5/8/19 10:37 AM, bosch.thamm@xxxxxxxxxxxxxx wrote:

Hello,

 

In a current project we are using geomesa-fs-data-store for our network-measurements. We are collecting these network-measurements from driving vehicles.

 

The issues is that we are facing a mismatch with returned data between “geomesa-fs export” and querying geomesa programmatically.

This is the geomesa-fs export-command: ./geomesa-fs export -p ………./geomesa-data-store/ -f ModemState --cql "dtg>=2019-04-13T23:40:00Z AND dtg<=2019-04-14T01:00:00Z AND vin LIKE '%VINSYSTEMTEST1%'"

 

This is the query from a debug-statement when querying geomesa programmatically:

2019-05-08 16:16:16.073 [http-nio-8130-exec-2] INFO  org.apache.parquet.filter2.compat.FilterCompat - Filtering using predicate: and(and(and(and(and(gteq(geom.x, -180.0), gteq(geom.y, -90.0)), lteq(geom.x, 180.0)), lteq(geom.y, 90.0)), and(gteq(dtg, 1555198800000), lteq(dtg, 1555203600000))), eq(vin, Binary{"VINSYSTEMTEST1"}))

 

While the export returns 1 event, the programmatic query returns 0 events.

The event is found in the programmatic case when extending the search-interval to an earlier time. E.g. from 2019-04-13T23:40:00Z to 2019-04-13T00:40:00Z

 

I am also attaching the test-data.

 

We are using:

Geomesa-fs-datastore_2.11:2.2.0

Geomesa-fs-storage-parquet_2.11:2.2.0

Partitioning-Scheme: daily, z2-2bit

 

The storage-type is parquet on AWS S3.

 

I hope you can help. Thanks,

Markus Thamm


If you are not the addressee, please inform us immediately that you have received this e-mail by mistake, and delete it. We thank you for your support.




_______________________________________________
geomesa-users mailing list
geomesa-users@xxxxxxxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.locationtech.org/mailman/listinfo/geomesa-users

 

Default Disclaimer Daimler AG
If you are not the addressee, please inform us immediately that you have received this e-mail by mistake, and delete it. We thank you for your support.


_______________________________________________
geomesa-users mailing list
geomesa-users@xxxxxxxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.locationtech.org/mailman/listinfo/geomesa-users