When using the library by running up and down the parse tree and extracting used columns and attributes of the parsed query I frequently ran into inexplicable errors and eventually into segmentation faults that crashed my python program. When going for a proof of concept I ended with the following code that only works for me with iPython. However, I am pretty sure that the problem is not iPython but sqlparser as segmentation faults occurred on multiple occasions when using the library and when doing the proof of concept I was able to eliminate everything but sqlparser related code - however the overall error changed to the GC problem.
I am using:
- an Ubuntu 14.04
- Python 2.7.6 (default, Oct 26 2016, 20:30:19)
- IPython 1.2.1
The code:
import sqlparser
def analyze_result_column_list(node):
ret = []
for subnode in node.list:
if subnode.expr.leftOperand is not None:
operand = subnode.expr.leftOperand
for item in operand.caseExpression.whenClauseItemList:
ret.append(item.comparison_expr.leftOperand.objectOperand.get_text())
return ret
def analyze_select_statement(root):
result_attributes = analyze_result_column_list(root.resultColumnList)
return [], []
class SqlQuery():
_query_string = None
_defines = None
_uses = None
def __init__(self, stmt, vendor=sqlparser.gsp_dbvendor.dbvmysql):
root = stmt.get_root()
self._uses, self._defines = analyze_select_statement(root)
parser = sqlparser.Parser()
query = "SELECT (CASE WHEN password = MD5( 'bitnami ') THEN 1 ELSE 0 END) AS zend_auth_credential_match FROM si_user WHERE (email = '[email protected] ')"
parser.check_syntax(query)
stmt = parser.get_statement(0)
pq = SqlQuery(stmt)
pq._defines
pq._uses
print "This was a triump,h \nI make a note here: \nHUGE SUCCESS."
Removing self._uses, self._defines = analyze_select_statement(root) leads to the error not manifesting.
The bug can be recreated using a fresh virtual environment and then installing nothing but the sqlparser ,as described in the readme. Then do the following steps:
- activate the virtenv
- start IPython
- mark and copy the above code
- type %paste
Python 2.7.6 (default, Oct 26 2016, 20:30:19)
Type "copyright", "credits" or "license" for more information.
IPython 1.2.1 -- An enhanced Interactive Python.
? -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help -> Python's own help system.
object? -> Details about 'object', use 'object??' for extra details.
In [1]: %paste
import sqlparser
def analyze_result_column_list(node):
ret = []
for subnode in node.list:
if subnode.expr.leftOperand is not None:
operand = subnode.expr.leftOperand
for item in operand.caseExpression.whenClauseItemList:
ret.append(item.comparison_expr.leftOperand.objectOperand.get_text())
return ret
def analyze_select_statement(root):
result_attributes = analyze_result_column_list(root.resultColumnList)
return [], []
class SqlQuery():
_query_string = None
_defines = None
_uses = None
def __init__(self, stmt, vendor=sqlparser.gsp_dbvendor.dbvmysql):
root = stmt.get_root()
self._uses, self._defines = analyze_select_statement(root)
parser = sqlparser.Parser()
query = "SELECT (CASE WHEN password = MD5( 'bitnami ') THEN 1 ELSE 0 END) AS zend_auth_credential_match FROM si_user WHERE (email = '[email protected] ')"
parser.check_syntax(query)
stmt = parser.get_statement(0)
pq = SqlQuery(stmt)
pq._defines
pq._uses
## -- End pasted text --
Fatal Python error: GC object already tracked
Aborted (core dumped)
If any additional feedback/versions/etc. are needed please notify me as I am happy to help in any way possible to get to the root of this issue.
When using the library by running up and down the parse tree and extracting used columns and attributes of the parsed query I frequently ran into inexplicable errors and eventually into segmentation faults that crashed my python program. When going for a proof of concept I ended with the following code that only works for me with iPython. However, I am pretty sure that the problem is not iPython but sqlparser as segmentation faults occurred on multiple occasions when using the library and when doing the proof of concept I was able to eliminate everything but sqlparser related code - however the overall error changed to the GC problem.
I am using:
The code:
Removing
self._uses, self._defines = analyze_select_statement(root)leads to the error not manifesting.The bug can be recreated using a fresh virtual environment and then installing nothing but the sqlparser ,as described in the readme. Then do the following steps:
If any additional feedback/versions/etc. are needed please notify me as I am happy to help in any way possible to get to the root of this issue.