Is this some byte order mark problem

Question

My question seems to come from differences between text files in different OSes. So I will try my luck here.

I would like to try the default project created from a .NET Core template on Ubuntu 18.04. I have installed SQL Server mssql-server_15.0.4003.23-3_amd64.deb and choose Express as edition. Since LocalDB is Windows only, I figure that I have to modify src/WebUI/appsettings.json to replace LocalDB connection string with the following SQL Server connection string (I am not sure if it is correct):

  "ConnectionStrings": {
      "DefaultConnection": "Server=localhost\\SQLEXPRESS,1433;Database=ExampleDb;User Id=sa;Password=password;"
  },

Then I go into src/WebUI/ and build it but fail (see below). The errors seem not related to my change of the connection string.

Regarding the first error, I open src/Domain/Domain.csproj in Emacs, which shows something strange <U+FEFF>, which https://en.wikipedia.org/wiki/Byte_order_mark says is about endianness. I wonder how I could make the files work?

<U+FEFF>namespace default.Domain.Enums
{
    public enum PriorityLevel
    {
        None,
        Low,
        Medium,
        High
    }
}

Thanks.

$ dotnet build
Microsoft (R) Build Engine version 16.4.0+e901037fe for .NET Core
Copyright (C) Microsoft Corporation. All rights reserved.

  Restore completed in 147.56 ms for /tmp/test/default/src/Domain/Domain.csproj.
  Restore completed in 213.91 ms for /tmp/test/default/src/Application/Application.csproj.
  Restore completed in 57.04 ms for /tmp/test/default/src/Infrastructure/Infrastructure.csproj.
  Restore completed in 9.65 ms for /tmp/test/default/src/WebUI/WebUI.csproj.
Enums/PriorityLevel.cs(1,11): error CS1001: Identifier expected [/tmp/test/default/src/Domain/Domain.csproj]
Enums/PriorityLevel.cs(1,26): error CS0116: A namespace cannot directly contain members such as fields or methods [/tmp/test/default/src/Domain/Domain.csproj]
Enums/PriorityLevel.cs(1,11): error CS1514: { expected [/tmp/test/default/src/Domain/Domain.csproj]
Enums/PriorityLevel.cs(1,18): error CS1022: Type or namespace definition, or end-of-file expected [/tmp/test/default/src/Domain/Domain.csproj]
Enums/PriorityLevel.cs(2,1): error CS1022: Type or namespace definition, or end-of-file expected [/tmp/test/default/src/Domain/Domain.csproj]
Exceptions/AdAccountInvalidException.cs(3,11): error CS1001: Identifier expected [/tmp/test/default/src/Domain/Domain.csproj]
Exceptions/AdAccountInvalidException.cs(3,26): error CS0116: A namespace cannot directly contain members such as fields or methods [/tmp/test/default/src/Domain/Domain.csproj]
Exceptions/AdAccountInvalidException.cs(3,11): error CS1514: { expected [/tmp/test/default/src/Domain/Domain.csproj]
Exceptions/AdAccountInvalidException.cs(3,18): error CS1022: Type or namespace definition, or end-of-file expected [/tmp/test/default/src/Domain/Domain.csproj]
Exceptions/AdAccountInvalidException.cs(4,1): error CS1022: Type or namespace definition, or end-of-file expected [/tmp/test/default/src/Domain/Domain.csproj]
ValueObjects/AdAccount.cs(1,7): error CS1001: Identifier expected [/tmp/test/default/src/Domain/Domain.csproj]
ValueObjects/AdAccount.cs(1,7): error CS1002: ; expected [/tmp/test/default/src/Domain/Domain.csproj]
ValueObjects/AdAccount.cs(1,22): error CS0116: A namespace cannot directly contain members such as fields or methods [/tmp/test/default/src/Domain/Domain.csproj]
ValueObjects/AdAccount.cs(1,7): error CS1022: Type or namespace definition, or end-of-file expected [/tmp/test/default/src/Domain/Domain.csproj]
ValueObjects/AdAccount.cs(1,28): error CS1022: Type or namespace definition, or end-of-file expected [/tmp/test/default/src/Domain/Domain.csproj]
ValueObjects/AdAccount.cs(2,7): error CS1001: Identifier expected [/tmp/test/default/src/Domain/Domain.csproj]
ValueObjects/AdAccount.cs(2,7): error CS1002: ; expected [/tmp/test/default/src/Domain/Domain.csproj]
ValueObjects/AdAccount.cs(2,22): error CS0116: A namespace cannot directly contain members such as fields or methods [/tmp/test/default/src/Domain/Domain.csproj]
ValueObjects/AdAccount.cs(2,7): error CS1022: Type or namespace definition, or end-of-file expected [/tmp/test/default/src/Domain/Domain.csproj]
ValueObjects/AdAccount.cs(2,32): error CS1022: Type or namespace definition, or end-of-file expected [/tmp/test/default/src/Domain/Domain.csproj]
ValueObjects/AdAccount.cs(6,11): error CS1001: Identifier expected [/tmp/test/default/src/Domain/Domain.csproj]
ValueObjects/AdAccount.cs(6,26): error CS0116: A namespace cannot directly contain members such as fields or methods [/tmp/test/default/src/Domain/Domain.csproj]
ValueObjects/AdAccount.cs(6,11): error CS1514: { expected [/tmp/test/default/src/Domain/Domain.csproj]
ValueObjects/AdAccount.cs(6,18): error CS1022: Type or namespace definition, or end-of-file expected [/tmp/test/default/src/Domain/Domain.csproj]
ValueObjects/AdAccount.cs(7,1): error CS1022: Type or namespace definition, or end-of-file expected [/tmp/test/default/src/Domain/Domain.csproj]
Common/ValueObject.cs(4,11): error CS1001: Identifier expected [/tmp/test/default/src/Domain/Domain.csproj]
Common/ValueObject.cs(4,26): error CS0116: A namespace cannot directly contain members such as fields or methods [/tmp/test/default/src/Domain/Domain.csproj]
Common/ValueObject.cs(4,11): error CS1514: { expected [/tmp/test/default/src/Domain/Domain.csproj]
Common/ValueObject.cs(4,18): error CS1022: Type or namespace definition, or end-of-file expected [/tmp/test/default/src/Domain/Domain.csproj]
Common/ValueObject.cs(5,1): error CS1022: Type or namespace definition, or end-of-file expected [/tmp/test/default/src/Domain/Domain.csproj]
Entities/TodoItem.cs(1,7): error CS1001: Identifier expected [/tmp/test/default/src/Domain/Domain.csproj]
Entities/TodoItem.cs(1,7): error CS1002: ; expected [/tmp/test/default/src/Domain/Domain.csproj]
Entities/TodoItem.cs(1,22): error CS0116: A namespace cannot directly contain members such as fields or methods [/tmp/test/default/src/Domain/Domain.csproj]
Entities/TodoItem.cs(1,7): error CS1022: Type or namespace definition, or end-of-file expected [/tmp/test/default/src/Domain/Domain.csproj]
Entities/TodoItem.cs(1,28): error CS1022: Type or namespace definition, or end-of-file expected [/tmp/test/default/src/Domain/Domain.csproj]
Entities/TodoItem.cs(2,7): error CS1001: Identifier expected [/tmp/test/default/src/Domain/Domain.csproj]
Entities/TodoItem.cs(2,7): error CS1002: ; expected [/tmp/test/default/src/Domain/Domain.csproj]
Entities/TodoItem.cs(2,22): error CS0116: A namespace cannot directly contain members such as fields or methods [/tmp/test/default/src/Domain/Domain.csproj]
Entities/TodoItem.cs(2,7): error CS1022: Type or namespace definition, or end-of-file expected [/tmp/test/default/src/Domain/Domain.csproj]
Entities/TodoItem.cs(2,27): error CS1022: Type or namespace definition, or end-of-file expected [/tmp/test/default/src/Domain/Domain.csproj]
Entities/TodoItem.cs(5,11): error CS1001: Identifier expected [/tmp/test/default/src/Domain/Domain.csproj]
Entities/TodoItem.cs(5,26): error CS0116: A namespace cannot directly contain members such as fields or methods [/tmp/test/default/src/Domain/Domain.csproj]
Entities/TodoItem.cs(5,11): error CS1514: { expected [/tmp/test/default/src/Domain/Domain.csproj]
Entities/TodoItem.cs(5,18): error CS1022: Type or namespace definition, or end-of-file expected [/tmp/test/default/src/Domain/Domain.csproj]
Entities/TodoItem.cs(6,1): error CS1022: Type or namespace definition, or end-of-file expected [/tmp/test/default/src/Domain/Domain.csproj]
Entities/TodoList.cs(1,7): error CS1001: Identifier expected [/tmp/test/default/src/Domain/Domain.csproj]
Entities/TodoList.cs(1,7): error CS1002: ; expected [/tmp/test/default/src/Domain/Domain.csproj]
Entities/TodoList.cs(1,22): error CS0116: A namespace cannot directly contain members such as fields or methods [/tmp/test/default/src/Domain/Domain.csproj]
Entities/TodoList.cs(1,7): error CS1022: Type or namespace definition, or end-of-file expected [/tmp/test/default/src/Domain/Domain.csproj]
Entities/TodoList.cs(1,28): error CS1022: Type or namespace definition, or end-of-file expected [/tmp/test/default/src/Domain/Domain.csproj]
Entities/TodoList.cs(4,11): error CS1001: Identifier expected [/tmp/test/default/src/Domain/Domain.csproj]
Entities/TodoList.cs(4,26): error CS0116: A namespace cannot directly contain members such as fields or methods [/tmp/test/default/src/Domain/Domain.csproj]
Entities/TodoList.cs(4,11): error CS1514: { expected [/tmp/test/default/src/Domain/Domain.csproj]
Entities/TodoList.cs(4,18): error CS1022: Type or namespace definition, or end-of-file expected [/tmp/test/default/src/Domain/Domain.csproj]
Entities/TodoList.cs(5,1): error CS1022: Type or namespace definition, or end-of-file expected [/tmp/test/default/src/Domain/Domain.csproj]
Common/AuditableEntity.cs(3,11): error CS1001: Identifier expected [/tmp/test/default/src/Domain/Domain.csproj]
Common/AuditableEntity.cs(3,26): error CS0116: A namespace cannot directly contain members such as fields or methods [/tmp/test/default/src/Domain/Domain.csproj]
Common/AuditableEntity.cs(3,11): error CS1514: { expected [/tmp/test/default/src/Domain/Domain.csproj]
Common/AuditableEntity.cs(3,18): error CS1022: Type or namespace definition, or end-of-file expected [/tmp/test/default/src/Domain/Domain.csproj]
Common/AuditableEntity.cs(4,1): error CS1022: Type or namespace definition, or end-of-file expected [/tmp/test/default/src/Domain/Domain.csproj]

Build FAILED.

Enums/PriorityLevel.cs(1,11): error CS1001: Identifier expected [/tmp/test/default/src/Domain/Domain.csproj]
Enums/PriorityLevel.cs(1,26): error CS0116: A namespace cannot directly contain members such as fields or methods [/tmp/test/default/src/Domain/Domain.csproj]
Enums/PriorityLevel.cs(1,11): error CS1514: { expected [/tmp/test/default/src/Domain/Domain.csproj]
Enums/PriorityLevel.cs(1,18): error CS1022: Type or namespace definition, or end-of-file expected [/tmp/test/default/src/Domain/Domain.csproj]
Enums/PriorityLevel.cs(2,1): error CS1022: Type or namespace definition, or end-of-file expected [/tmp/test/default/src/Domain/Domain.csproj]
Exceptions/AdAccountInvalidException.cs(3,11): error CS1001: Identifier expected [/tmp/test/default/src/Domain/Domain.csproj]
Exceptions/AdAccountInvalidException.cs(3,26): error CS0116: A namespace cannot directly contain members such as fields or methods [/tmp/test/default/src/Domain/Domain.csproj]
Exceptions/AdAccountInvalidException.cs(3,11): error CS1514: { expected [/tmp/test/default/src/Domain/Domain.csproj]
Exceptions/AdAccountInvalidException.cs(3,18): error CS1022: Type or namespace definition, or end-of-file expected [/tmp/test/default/src/Domain/Domain.csproj]
Exceptions/AdAccountInvalidException.cs(4,1): error CS1022: Type or namespace definition, or end-of-file expected [/tmp/test/default/src/Domain/Domain.csproj]
ValueObjects/AdAccount.cs(1,7): error CS1001: Identifier expected [/tmp/test/default/src/Domain/Domain.csproj]
ValueObjects/AdAccount.cs(1,7): error CS1002: ; expected [/tmp/test/default/src/Domain/Domain.csproj]
ValueObjects/AdAccount.cs(1,22): error CS0116: A namespace cannot directly contain members such as fields or methods [/tmp/test/default/src/Domain/Domain.csproj]
ValueObjects/AdAccount.cs(1,7): error CS1022: Type or namespace definition, or end-of-file expected [/tmp/test/default/src/Domain/Domain.csproj]
ValueObjects/AdAccount.cs(1,28): error CS1022: Type or namespace definition, or end-of-file expected [/tmp/test/default/src/Domain/Domain.csproj]
ValueObjects/AdAccount.cs(2,7): error CS1001: Identifier expected [/tmp/test/default/src/Domain/Domain.csproj]
ValueObjects/AdAccount.cs(2,7): error CS1002: ; expected [/tmp/test/default/src/Domain/Domain.csproj]
ValueObjects/AdAccount.cs(2,22): error CS0116: A namespace cannot directly contain members such as fields or methods [/tmp/test/default/src/Domain/Domain.csproj]
ValueObjects/AdAccount.cs(2,7): error CS1022: Type or namespace definition, or end-of-file expected [/tmp/test/default/src/Domain/Domain.csproj]
ValueObjects/AdAccount.cs(2,32): error CS1022: Type or namespace definition, or end-of-file expected [/tmp/test/default/src/Domain/Domain.csproj]
ValueObjects/AdAccount.cs(6,11): error CS1001: Identifier expected [/tmp/test/default/src/Domain/Domain.csproj]
ValueObjects/AdAccount.cs(6,26): error CS0116: A namespace cannot directly contain members such as fields or methods [/tmp/test/default/src/Domain/Domain.csproj]
ValueObjects/AdAccount.cs(6,11): error CS1514: { expected [/tmp/test/default/src/Domain/Domain.csproj]
ValueObjects/AdAccount.cs(6,18): error CS1022: Type or namespace definition, or end-of-file expected [/tmp/test/default/src/Domain/Domain.csproj]
ValueObjects/AdAccount.cs(7,1): error CS1022: Type or namespace definition, or end-of-file expected [/tmp/test/default/src/Domain/Domain.csproj]
Common/ValueObject.cs(4,11): error CS1001: Identifier expected [/tmp/test/default/src/Domain/Domain.csproj]
Common/ValueObject.cs(4,26): error CS0116: A namespace cannot directly contain members such as fields or methods [/tmp/test/default/src/Domain/Domain.csproj]
Common/ValueObject.cs(4,11): error CS1514: { expected [/tmp/test/default/src/Domain/Domain.csproj]
Common/ValueObject.cs(4,18): error CS1022: Type or namespace definition, or end-of-file expected [/tmp/test/default/src/Domain/Domain.csproj]
Common/ValueObject.cs(5,1): error CS1022: Type or namespace definition, or end-of-file expected [/tmp/test/default/src/Domain/Domain.csproj]
Entities/TodoItem.cs(1,7): error CS1001: Identifier expected [/tmp/test/default/src/Domain/Domain.csproj]
Entities/TodoItem.cs(1,7): error CS1002: ; expected [/tmp/test/default/src/Domain/Domain.csproj]
Entities/TodoItem.cs(1,22): error CS0116: A namespace cannot directly contain members such as fields or methods [/tmp/test/default/src/Domain/Domain.csproj]
Entities/TodoItem.cs(1,7): error CS1022: Type or namespace definition, or end-of-file expected [/tmp/test/default/src/Domain/Domain.csproj]
Entities/TodoItem.cs(1,28): error CS1022: Type or namespace definition, or end-of-file expected [/tmp/test/default/src/Domain/Domain.csproj]
Entities/TodoItem.cs(2,7): error CS1001: Identifier expected [/tmp/test/default/src/Domain/Domain.csproj]
Entities/TodoItem.cs(2,7): error CS1002: ; expected [/tmp/test/default/src/Domain/Domain.csproj]
Entities/TodoItem.cs(2,22): error CS0116: A namespace cannot directly contain members such as fields or methods [/tmp/test/default/src/Domain/Domain.csproj]
Entities/TodoItem.cs(2,7): error CS1022: Type or namespace definition, or end-of-file expected [/tmp/test/default/src/Domain/Domain.csproj]
Entities/TodoItem.cs(2,27): error CS1022: Type or namespace definition, or end-of-file expected [/tmp/test/default/src/Domain/Domain.csproj]
Entities/TodoItem.cs(5,11): error CS1001: Identifier expected [/tmp/test/default/src/Domain/Domain.csproj]
Entities/TodoItem.cs(5,26): error CS0116: A namespace cannot directly contain members such as fields or methods [/tmp/test/default/src/Domain/Domain.csproj]
Entities/TodoItem.cs(5,11): error CS1514: { expected [/tmp/test/default/src/Domain/Domain.csproj]
Entities/TodoItem.cs(5,18): error CS1022: Type or namespace definition, or end-of-file expected [/tmp/test/default/src/Domain/Domain.csproj]
Entities/TodoItem.cs(6,1): error CS1022: Type or namespace definition, or end-of-file expected [/tmp/test/default/src/Domain/Domain.csproj]
Entities/TodoList.cs(1,7): error CS1001: Identifier expected [/tmp/test/default/src/Domain/Domain.csproj]
Entities/TodoList.cs(1,7): error CS1002: ; expected [/tmp/test/default/src/Domain/Domain.csproj]
Entities/TodoList.cs(1,22): error CS0116: A namespace cannot directly contain members such as fields or methods [/tmp/test/default/src/Domain/Domain.csproj]
Entities/TodoList.cs(1,7): error CS1022: Type or namespace definition, or end-of-file expected [/tmp/test/default/src/Domain/Domain.csproj]
Entities/TodoList.cs(1,28): error CS1022: Type or namespace definition, or end-of-file expected [/tmp/test/default/src/Domain/Domain.csproj]
Entities/TodoList.cs(4,11): error CS1001: Identifier expected [/tmp/test/default/src/Domain/Domain.csproj]
Entities/TodoList.cs(4,26): error CS0116: A namespace cannot directly contain members such as fields or methods [/tmp/test/default/src/Domain/Domain.csproj]
Entities/TodoList.cs(4,11): error CS1514: { expected [/tmp/test/default/src/Domain/Domain.csproj]
Entities/TodoList.cs(4,18): error CS1022: Type or namespace definition, or end-of-file expected [/tmp/test/default/src/Domain/Domain.csproj]
Entities/TodoList.cs(5,1): error CS1022: Type or namespace definition, or end-of-file expected [/tmp/test/default/src/Domain/Domain.csproj]
Common/AuditableEntity.cs(3,11): error CS1001: Identifier expected [/tmp/test/default/src/Domain/Domain.csproj]
Common/AuditableEntity.cs(3,26): error CS0116: A namespace cannot directly contain members such as fields or methods [/tmp/test/default/src/Domain/Domain.csproj]
Common/AuditableEntity.cs(3,11): error CS1514: { expected [/tmp/test/default/src/Domain/Domain.csproj]
Common/AuditableEntity.cs(3,18): error CS1022: Type or namespace definition, or end-of-file expected [/tmp/test/default/src/Domain/Domain.csproj]
Common/AuditableEntity.cs(4,1): error CS1022: Type or namespace definition, or end-of-file expected [/tmp/test/default/src/Domain/Domain.csproj]
    0 Warning(s)
    60 Error(s)

Philip Couling · Answer 1 · 2020-01-17T09:23:18.193

In my opinion byte order marks are a bit of a mess and generally unnecessary in modern computing. Their original use was for UTF-16 and UTF-32 which, for interchang between computers of different architecture, needed a way to differentiate between little-endian and big-endian. UTF-16 and UTF-32 are much less favoured than UTF-8 these days and utf-8 doesn't need a BOM.

The standard on when to use a BOM is annoyingly vague:

See page 67 here: http://www.unicode.org/versions/Unicode12.0.0/UnicodeStandard-12.0.pdf

Data streams (or files) that begin with the U+FEFF byte order mark are likely to contain Unicode characters. It is recommended that applications sending or receiving untyped data streams of coded characters use this signature. If other signaling methods are used, signatures should not be employed.

As a general rule, most data streams are typed. For example utf-8 configuration files are expected to be utf-8 and so by definition are not "untyped". Thus as a general rule BOM should not be included unless you know otherwise.

You are more likely to cause problems by adding a BOM than removing one. UTF-8 was designed to be backwards compatible with ASCII. This allows you to use UTF-8 in a lot of places that were only expecting ASCII. But the BOM is not part of and not ASCII and many / most programs reading text files do not handle it well. Most don't know to ignore it at the start of a file, or strip it when concatenating files. Programs that expect a BOM are usually tolerant to its absence.

However many text editors will add a BOM without telling you. You may have inadvertently added it when you edited the configuration file.

There's another question directly asking how to remove a BOM: How can I remove the BOM from a UTF-8 file?

I would remove any BOM added to your configuration files. It is highly unlikely MS SQL is expecting them.

score 2 · Answer 2 · answered Jan 17 '20 at 09:27

2

Mayhap somthing like

iconv -fUTF16 -tUTF8 file

answered Jan 17 '20 at 09:27

RudiC

8,969

Is this some byte order mark problem

2 Answers2