I am in the process of setting up Presto and Redis on my local machine using the instructions provided in the Presto-Redis Documentation.
Problem Overview:
When I set
redis.key-prefix-schema-table=truedev:simple_table: for a Redis key (as recommended by the Presto Redis connector page), all key columns appear as null with the internal column _key_corrupt being true. However, the value columns are parsed correctly.
On the other hand, when I utilize
redis.key-prefix-schema-table=falseWhat's Working:
I have successfully set up an example involving a Redis server and a Presto server running on my local setup.
This is the content of `etc/catalog/redis.properties` file used by Presto:
connector.name=redis
redis.table-names=simple_table
redis.default-schema=dev
redis.nodes=127.0.0.1:6379
redis.table-description-dir=/Users/acarson/var/redis/tables
redis.key-delimiter=:
redis.key-prefix-schema-table=false
redis.hide-internal-columns=false
A table definition file exists at
/Users/acarson/var/redis/tables/simple_table.json{
"tableName": "simple_table",
"schemaName": "dev",
"key": {
"dataFormat": "json",
"fields": [
{
"name": "id",
"mapping": "id",
"type": "BIGINT"
},
{
"name": "word",
"mapping": "word",
"type": "VARCHAR"
}
]
},
"value": {
"dataFormat": "json",
"fields": [
{
"name": "name",
"mapping": "name",
"type": "VARCHAR"
},
{
"name": "number",
"mapping": "number",
"type": "BIGINT"
},
{
"name": "boolean",
"mapping": "boolean",
"type": "BOOLEAN"
}
]
}
}
Using redis-cli, I insert a value using the following command:
SET '{"id": 42, "word": "foo"}' '{"name": "bar", "number": 3, "boolean": "false"}'
Subsequently, I'm able to query the data using the Presto CLI and the resulting columns display as expected:
presto:dev> SELECT * FROM simple_table;
id | word | name | number | boolean | _key | _value | _key_length | _value_length | _key_corrupt | _value_corrupt
----+------+------+--------+---------+---------------------------+--------------------------------------------------+-------------+---------------+--------------+----------------
42 | foo | bar | 3 | false | {"id": 42, "word": "foo"} | {"name": "bar", "number": 3, "boolean": "false"} | 25 | 48 | false | false
Issue Encountered:
Upon changing the value of redis.key-prefix-schema-table to true in redis.properties, restarting Presto, clearing all keys in redis-cli with FLUSHALL, and inserting a new row using the schema table prefix:
SET 'dev:simple_table:{"id": 42, "word": "foo"}' '{"name": "bar", "number": 3, "boolean": "false"}'
Querying from Presto CLI reveals that _key_corrupt=true and the key values are null, despite the values working fine.
presto:dev> SELECT * FROM simple_table;
id | word | name | number | boolean | _key | _value | _key_length | _value_length | _key_corrupt | _value_corrupt
------+------+--------+--------+---------+-----------------------------------------------+-----------------------------------------------------+-------------+---------------+--------------+----------------
NULL | NULL | bar | 3 | false | dev:simple_table:{"id": 42, "word": "foo"} | {"name": "bar", "number": 3, "boolean": "false"} | 42 | 48 | true | false
I have examined the log levels of both Redis and Presto for clues, but so far, nothing indicates any errors or explanations for why the key might be corrupt. It seems like there could be an issue with my syntax for defining Redis keys, even though I am adhering to the exact method outlined in the Presto Redis documentation and also referring to the Presto-Redis source code which specifies keys should follow the format "schema:table:*" with this flag.
Here are excerpts from the Presto logs at debug level during the query execution:
2016-10-20T17:09:55.048-0700 INFO main com.facebook.presto.server.PrestoServer ======== SERVER STARTED ========
(Additional log entries as described were deleted for brevity)