Archive

Posts Tagged ‘Parser’

Abstract Syntax Tree (AST) Parser for C/C++

December 24th, 2009 NamPham No comments

Way, after four hours reading, searching, andcoding, I finally understand how to use Eclipse C/C++ Development Tooling (CDT) to parse the C/C++ source code. In detail, it used the Document Object Model (DOM) to parse and update AST whenever there is a change in source file.

The reason I spent so much time solving this problem because of lack in APIs documentation and also I thought that the sample code using CDT is from old version (2.0) in which many APIs were deprecated.

Similar to JDT- the tool parse Java source code to build AST – CDT works in similar steps:

  1. Create an IASTTranslationUnit instance
  2. Overide abstract class ASTVisior
  3. Traverse the IASTTranslationUnit using ASTVisior

Here are the sample code which works with Eclipse SDK 3.5.1 and CDT version 6.0.1:

Import:

?View Code JAVA
1
2
3
4
5
6
7
8
9

import org.eclipse.cdt.core.dom.ICodeReaderFactory;
import org.eclipse.cdt.core.dom.ast.ASTVisitor;
import org.eclipse.cdt.core.dom.ast.IASTTranslationUnit;
import org.eclipse.cdt.core.dom.ast.gnu.cpp.GPPLanguage;
import org.eclipse.cdt.core.parser.CodeReader;
import org.eclipse.cdt.core.parser.DefaultLogService;
import org.eclipse.cdt.core.parser.IParserLogService;
import org.eclipse.cdt.core.parser.IScannerInfo;
import org.eclipse.cdt.core.parser.ScannerInfo;

Main program:

?View Code JAVA
1
2
3
4
5
6
7
8
9
10
11
12
13

IParserLogService log = new DefaultLogService();
char[] code = readFile("C:\\grep.cpp");
CodeReader reader = new CodeReader(code);
Map definedSymbols = new HashMap();
String[] includePaths = new String[0];
IScannerInfo info = new ScannerInfo(definedSymbols, includePaths);
ICodeReaderFactory readerFactory = FileCodeReaderFactory.getInstance();
 
IASTTranslationUnit translationUnit = GPPLanguage.getDefault()
.getASTTranslationUnit(reader, info, readerFactory, null, log);
 
ASTVisitor visitor = new ImplASTVisitor();
translationUnit.accept(visitor);

Please note that I already override the ASTVisior class with my own one ImplASTVisitor:

?View Code JAVA
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46

public class ImplASTVisitor extends ASTVisitor {
	public ImplASTVisitor(){
		super.shouldVisitNames = true;
		super.shouldVisitDeclarations = true;
		super.shouldVisitInitializers = true;
		super.shouldVisitParameterDeclarations = true;
		super.shouldVisitDeclarators = true;
		super.shouldVisitDeclSpecifiers = true;
		super.shouldVisitExpressions = true;
		super.shouldVisitStatements = true;
		super.shouldVisitTypeIds = true;
		super.shouldVisitEnumerators = true;
		super.shouldVisitTranslationUnit = true;
		super.shouldVisitProblems = true;
	}
 
	/**
	 * @return continue to continue visiting, abort to stop, skip to not descend
	 *         into this node.
	 */
	public final static int PROCESS_SKIP = 1;
	public final static int PROCESS_ABORT = 2;
	public final static int PROCESS_CONTINUE = 3;
	/**
	 *
	 * visit methods
	 *
	 */
	private void print(IASTNode node){
		System.out.println(node.getRawSignature());
	}
	public int visit(IASTTranslationUnit tu) {
		print(tu);
		return PROCESS_CONTINUE;
	}
	public int visit(IASTName name) {
		print(name);
		return PROCESS_CONTINUE;
	}
 
	public int visit(IASTDeclaration declaration) {
		print(declaration);
		return PROCESS_CONTINUE;
	}
       .....
}

I actually did not write all this portion of code. I read, understand and rewrite it based on the following sources:

  • Thanks Jason Montojo for this sample source code: Nabble
Categories: Algorithms, Research Tags: , , , ,