SQLite Forum

how DISTINCT remove the duplicate rows?
Login
It is pretty simple.

1.  For each projection candidate generate a row/key structure.  
2.  If the row/key exists in the ephemeral filter table go to step 5.  
3.  Add the row/key to the ephemeral filter table.  
4.  Output the candidate row.  
5.  If we have not reached the end of the candidates, go to step 1 to process the next candidate.   
6.  Clean up and discard the ephemeral table and halt.  

You will note that this is *exactly* the procedure that SQLite3 uses to process the statement:

```
sqlite> CREATE TABLE tab(c);
sqlite> INSERT INTO tab values (1), (1.0);
sqlite> .eqp full
sqlite> SELECT DISTINCT c FROM tab;
QUERY PLAN
|--SCAN tab (~1048576 rows)
`--USE TEMP B-TREE FOR DISTINCT
addr  opcode         p1    p2    p3    p4             p5  comment
----  -------------  ----  ----  ----  -------------  --  -------------
0     Init           0     17    0                    0   Start at 17
1     OpenEphemeral  1     0     0     k(1,B)         8   nColumn=0
2     OpenRead       0     2     0     1              0   root=2 iDb=0; tab
3     ColumnsUsed    0     0     0     1              0
4     Explain        4     0     0     SCAN tab (~1048576 rows)  0
5     Noop           0     0     0                    0   Begin WHERE-loop0: tab
6     Rewind         0     15    0                    0
7       Noop           0     0     0                    0   Begin WHERE-core
8       Column         0     0     1                    0   r[1]=tab.c
9       Found          1     14    1     1              0   key=r[1]
10      MakeRecord     1     1     2                    0   r[2]=mkrec(r[1])
11      IdxInsert      1     2     1     1              16  key=r[2]
12      ResultRow      1     1     0                    0   output=r[1]
13      Noop           0     0     0                    0   End WHERE-core
14    Next           0     7     0                    1
15    Noop           0     0     0                    0   End WHERE-loop0: tab
16    Halt           0     0     0                    0
17    Transaction    0     0     1     0              1   usesStmtJournal=0
18    Goto           0     1     0                    0
┌───┐
│ c │
├───┤
│ 1 │
└───┘
```