Skip to content

KustoCode.ParseAndAnalyze does not parse RealLiteralExpressions in non-english cultures #197

Description

The input
print 123.456
will generate a parse-tree with a LiteralExpression whose LiteralValue property contains 123456 if the culture is set to Greek. If the culture is set to french, it will generate a LiteralValue of 0

This seems incorrect; the documentation I've found implies that KQL always uses invariant-culture for number parsing. Note, this is not problem with the print operator; the same issue can be demonstrated with

datatable(A:real)[123.456,789]

The following code can be used to demonstrate the problem

using System.Globalization;
using System.Text;
using Kusto.Language;
using Kusto.Language.Symbols;
using Kusto.Language.Syntax;

var cultureName = "el-GR"; //problem found with greek culture but also demonstrated with french
CultureInfo.CurrentUICulture = CultureInfo.CurrentCulture = new CultureInfo(cultureName);

var query = "print 123.456";

var kustoCode = KustoCode.ParseAndAnalyze(query, GlobalState.Default);
DumpKustoTree(kustoCode);
return;

void DumpKustoTree(KustoCode tree)
{
    var sb = new StringBuilder();
    var indent = 0;
    SyntaxElement.WalkNodes(
        tree.Syntax,
        node =>
        {
            sb.Append(new string(' ', indent));
            sb.AppendLine(
                $"{node.Kind}: {node.ToString(IncludeTrivia.SingleLine)}: {SchemaDisplay.GetText((node as Expression)?.ResultType)}");
            if (node is LiteralExpression literal)
            {
                var value = literal.LiteralValue;
                var v = value is IFormattable f
                    ? f.ToString(null, CultureInfo.InvariantCulture)
                    : value?.ToString() ?? "<null>";
                sb.AppendLine(new string(' ', indent) + $"LITERAL {value!.GetType().Name}  {v}");
            }

            indent++;
        },
        _ => { indent--; });

    Console.WriteLine(sb.ToString());
}

This generates the following output:

QueryBlock: print 123.456:
 List: :
 List: print 123.456:
  SeparatedElement: print 123.456:
   ExpressionStatement: print 123.456:
    PrintOperator: print 123.456: (print_0: real)
     List: 123.456:
      SeparatedElement: 123.456:
       RealLiteralExpression: 123.456: real
       LITERAL Double  123456

The problem occurs as far back as version 11.0.0 of the library.

It's possible that there is some hidden setup or parameter I'm missing that would force the use of invariant culture, other than that the best workarounds would seem to be to force the current culture to Invariant while parsing or perhaps to ignore the LiteralValue in the property and parse it out manually from the LiteralText using invariantculture.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions