one small technical clarification:

Your description might be misinterpreted as meaning that anything other than an INTEGER PRIMARY KEY will duplicate index data (not exactly true for ALL cases, see below).

Referencing the example from https://www.sqlite.org/withoutrowid.html :
----------
...Because there is only a single B-Tree, the text of the "word" column is only stored once in the database. Furthermore, querying the "cnt" value for a specific "word" only involves a single binary search into the main B-Tree, since the "cnt" value can be retrieved directly from the record found by that first search and without the need to do a second binary search on the rowid.
-------------
This is why a WITHOUT ROWID table must have a UNIQUE PRIMARY KEY (even if it's a compound key).  The B-Tree requires something to traverse.

Expanding the OP question a bit:

There will be other characteristics (such as insert/delete/update performance ) in  having longer unique primary key (as noted in the reference docs cited).  Given that the OP data structure requires a bit more thought, they would be well advised to consider population and maintenance in their review as well.

As explained in the docs here (https://www.sqlite.org/withoutrowid.html), you will need some sort of unique key on a table, whether it's ROWID, or your own unique primary key.

create table t(value int) strict; -- using strict to make sure we keep value an int
create index value_dups_on_t on t (value);
insert into t values (1), (2), (1);
select * from t;
┌───────┐
│ value │
├───────┤
│ 1     │
│ 2     │
│ 1     │
└───────┘

That creates about the smallest non unique integer values in a table structure for sqlite I am aware of for a unary relation with duplicates.

It's not clear of it's usefulness though, as a sorted list in memory would allow  a binary search without the overhead of sql, so if this is disconnected from any other data (a unary relation) it's a bit curious as to why duplicates are necessary.

If one had meaningful duplicates, how would you know if it's the first, or second, or nth value? and if that does not matter, why have duplicates (and then you can use value as integer primary key again, not to mention a small table)?

If this table is of the unary relation type (just looks up whether it's in the table) perhaps look at the insert operation of the table and filter out duplicates at that point?

Without knowing the rest of the parts it is a bit difficult to guess which way you should go with this.

I have a  "value", it is not unique, but it is the value that I do equal query and joins on it, a lot.
I know SQLite needs a unique key for its internal structure and it is no problem, ROWID was hidden in the table and that is okay. The extra size of ROWID is not my problem.

My problem is that, to find "value" faster, I need to make an index, and that index, duplicates the ROWID, value duo if I am not mistaken, at list duplicating the database size suggests that.
As far as I know, the index is an internal table, with ROWID and value, sorted by value, but not unique.

So my question is that if there is way I can have that internal table, with a sorted "value" so I can query it much faster (as SQLite uses binary search or similar on these sorted columns)?

I am aware that if you have 20 column table, duplicating ROWID and "value", is not much problem. But my table is hundreds of million of just these two values, and I naturally prefer too look for a way to tell SQLite: "Hey can you keep value as sorted so you can search it faster?"
And I guess it can not, but I wanted to make sure to ask.

1. Insert value + delimiter + next highest record counter
2. Search using value + delimiter
Hope this helps.  Tested using explain query plan and index is used.

drop table if exists t1;
create table if not exists t1(value text primary key) without rowid;
insert into t1 values('1312220-1');
insert into t1 values('1312220-2');
insert into t1 values('1-3');

select * from t1 where value >= '1-' and value <= '1-'||'A'; 
explain query plan select * from t1 where value >= '1-' and value <= '1-'||'A';

select * from t1 where value >= '1312220-' and value <= '1312220-'||'A'; 
explain query plan select * from t1 where value >= '1312220-' and value <= '1312200-'||'A';

SEARCH t1 USING PRIMARY KEY (value>? AND value<?)

Could also use values like this:
1312220
0000001 or select printf('%07d', '1')

Save comma delimited pointers in second column of each unique value.
If there many duplicates this will greatly reduce the file size.

value  addr
A      1,4,7
B      2,5,8
C      3,6,9

FUNCTION PBMAIN AS LONG

 create table if not exists t1(value text primary key,addr text) without rowid
 create table if not exists counttable(counter integer)

 begin exclusive
 sCounter = FORMAT$(VAL(slSelStr("select counter from counttable limit 1"))+1)
 IF sCounter <> "1" THEN
  update counttable set counter=" + sCounter
 ELSE
  insert into counttable values(" + sCounter + ")"
 END IF
 char = 65
 sValue = CHR$(char)
 select * from t1 where value='"+sValue + "'"

 IF slGetRow THEN
  update t1 set addr ='" + slf(2) + "," + sCounter + "'  where value='"+sValue + "'"
 ELSE
  insert into t1 values('" + sValue +  "'," + sCounter + ")"
 END IF
 commit
END FUNCTION

Added binding so input keys can be phrases that include single quotes

#INCLUDE ONCE "sqlitening.inc"  'Anything.bas
MACRO bind(str) = slBuildbindDat(str,"T")

FUNCTION PBMAIN AS LONG
 LOCAL sWord,sdata AS STRING
 slOpen "words.db3","C"
 'slexe  "drop table if exists WordTable;drop table if exists PointerTable" 'start over
 slexe  "create table if not exists WordTable(Word text primary key,Pointers text) without rowid"
 slexe  "create table if not exists PointerTable(HighPointer integer)
 IF LEN(slSelStr("Select * from PointerTable"))=0 THEN slexe "insert into PointerTable values(0)"
 DO
  sdata = GetRecordSet("Select * from WordTable")
  sWord = INPUTBOX$(sdata,"Insert word",sWord)
  IF LEN(sWord) = 0 THEN EXIT DO
  slexe "begin immediate"
  AddWord sWord
  slexe "end"
 LOOP
END FUNCTION
'=====================================================================================
FUNCTION AddWord(sWord AS STRING) AS STRING
 LOCAL sHighPointer AS STRING

 'increment pointertable.highpointer
 sHighPointer = FORMAT$(VAL(slSelStr("Select HighPointer from PointerTable limit 1"))+1)
 slexebind  "update PointerTable set HighPointer=?",bind(sHighPointer)

 'search wordtable.word for sWord
 slSelBind  "Select Word,Pointers from WordTable where Word=?", bind(sWord)

 IF slGetRow THEN 'update wordtable pointers column
  slexebind "update WordTable set Pointers = ? where Word = ?",bind(slf(2)+ "," + sHighPointer) + bind(sWord)
 ELSE             'insert wordtable
  slexebind "insert into WordTable values(?,?)",bind(sWord)+bind(sHighPointer)
 END IF
END FUNCTION
'=====================================================================================
FUNCTION GetRecordSet(sql AS STRING) AS STRING
 LOCAL sArray() AS STRING
 slSelAry sql,sArray(),"Q9"
 FUNCTION = JOIN$(sArray(),$CRLF)
END FUNCTION