Within a token that is not quoted, it is your shell that will perform
expansion, not the command that you are running.
This means that, when you enter find . -name "*.txt" -print, then
find receives the literal *.txt as one of its parameters, and uses
this pattern as the argument to its -name option, which will match
the names of files found against it before applying -print.
On the other hand, when you enter find . -name *.txt -print, the
shell passes the expanded version of *.txt to find. Several cases
are possible:
There are no files matching *.txt in the current directory: find
receives a literal *.txt (assuming default bash settings);
there is exactly one file matching *.txt in the current
directory; let's say it is a.txt: find receives this file name,
and matches all files named a.txt found starting at the current
directory;
several files match *.txt in the current directory (this appears
to be your case): -name receives the first one as its parameter,
and the others are further path parameters to find, which complains
about not being given all paths before the expression.
This is the expected behavior.
Let's assume the following file hierarchy:
.
├── a.txt
├── b.txt
├── c.txt
└── foo
├── a.txt
├── b.txt
└── c.txt
The actual parameters that find receives in eah case can be observed
by replacing the call to find with printf '%s\n', which will print
each expanded argument on its own line:
$ printf '%s\n' . -name "*.txt" -print
.
-name
*.txt
-print
$ printf '%s\n' . -name *.txt -print
.
-name
a.txt
b.txt
c.txt
-print
As you can see, the second invokation that you posted is equivalent,
given the existing files, to find . -name a.txt b.txt c.txt -print.