Bilal Hatipoglu's Blog

Matematik Asla Yalan Söylemez!

Bilal Hatipoglu — 12.02.2008 22:53:36

Feed users, click here to go to this topic's page. You can find related topics and reference links and you can read the comments for this topic

Feed users, click here to post a comment for this topic

Note: There might be problems about HTML code and parsing and displaying HTML code of your viewer application. Also, there are problems caused by Google Reader on displaying of code samples on the entry. Also, the entry might be updated. So, it is recommended to visit the page of the entry.

Finding and Removing Loop on a Singly-Linked List

Bilal Hatipoglu — 06.02.2008 12:02:52

One of the most common interview questions for software professionals is "How do you find a loop in a singly linked list?". Most of the people tend to think in the recursive way to solve this problem. The truth is that the most optimal solution for this problem lies out of the scope of the linked list.

A singly linked list is a common data structure familiar to all computer scientists. A singly linked list is made of nodes where each node has a pointer to the next node (or null to end the list). On a linked list, adding a new fist element, removing the existing first element, and examining the first element are very fast O(1) operations.

When working with singly linked list, you are typically given a link to the first node. Common operations on a singly linked list are iterating through all the nodes, adding to the list, or deleting from the list. Algorithms for these operations generally require a well formed linked list. That is a linked list without loops or cycles in it.

If a linked list has a cycle:

• The malformed linked list has no end (no node ever has a null next_node pointer)
• The malformed linked list contains two links to some node
• Iterating through the malformed linked list will yield all nodes in the loop multiple times

A malformed linked list with a loop causes iteration over the list to fail because the iteration will never reach the end of the list. Therefore, it is desirable to be able to detect that a linked list is malformed before trying an iteration. This article is a discussion of various algorithms to detect a loop in a singly linked list.

Incorrect "Solutions"

Mark Each Node

Traverse the list and mark each node as having been seen. If you come to a node that has already been marked, then you know that the list has a loop.

// Incorrect 'solution'
function boolean hasLoop(Node startNode){
  Node currentNode = startNode;
  do {
    if (currentNode.seen) return true;
    currentNode.seen = true;
  } while (currentNode = currentNode.next());
  return false;
}

The problem with this solution is ensuring that "seen" is marked as false for all the nodes before you start. If the linked list has a loop, it isn't possible to iterate over each node to set "seen" to "false" as an initial value for each node.

Even if you are able to solve the initial value problem, each node in a linked list may not have a field to use for this purpose. Requiring such a field in each node would mean that this is not a generic solution. As we will see later, this field in not needed for a perfectly correct and efficient solution anyway.

Detect Only Full Loops

When asked to come up with a solution, a common pitfall is not detecting all loops, but just a loop where the last node links to the first. A loop could still occur (and not be detected) if the last element linked to (for example) the second element.

// Incorrect 'solution'
function boolean hasLoop(Node startNode){
  Node currentNode = startNode;
  while (currentNode = currentNode.next()){
    if (currentNode == startNode) return true;
  }
  return false;
}

Most Common Inefficient Solution

Keep a hash set of all nodes seen so far (O(n) time complexity, O(n) space complexity)

Keeping a set of all the nodes have seen so far and testing to see if the next node is in that set would be a perfectly correct solution. It would run fast as well. However it would use enough extra space to make a copy of the linked list. Allocating that much memory is prohibitively expensive for large lists.

// Inefficient solution
function boolean hasLoop(Node startNode){
  HashSet nodesSeen = new HashSet();
  Node currentNode = startNode;
  do {
    if (nodesSeen.contains(currentNode)) return true;
    nodesSeen.add(currentNode);
  } while (currentNode = currentNode.next());
  return false;
}

Best Solutions

Catch Larger and Larger Loops (O(n) time complexity)

Always store some node to check. Occasionally reset this node to avoid the "Detect Only Full Loops" problem. When resetting it, double the amount of time before resetting it again.

// Good solution
function boolean hasLoop(Node startNode){
  Node currentNode = startNode;
  Node checkNode = null;
  int since = 0;
  int sinceScale = 2;
  do {
    if (checkNode == currentNode) return true;
    if (since >= sinceScale){
        checkNode = currentNode;
        since = 0;
        sinceScale = 2*sinceScale;
    }
    since++;
  } while (currentNode = currentNode.next());
  return false;
}

This solution is O(n) because sinceScale grows linearly with the number of calls to next(). Once sinceScale is greater than the size of the loop, another n calls to next() may be required to detect the loop. This solution requires up to 3 traversals of the list.

This solution was devised by Stephen Ostermiller and proven O(n) by Daniel Martin.

Catch Loops in Two Passes (O(n) time complexity)

Simultaneously go through the list by ones (slow iterator) and by twos (fast iterator). If there is a loop the fast iterator will go around that loop twice as fast as the slow iterator. The fast iterator will lap the slow iterator within a single pass through the cycle. Detecting a loop is then just detecting that the slow iterator has been lapped by the fast iterator.

// Best solution
function boolean hasLoop(Node startNode){
  Node slowNode = Node fastNode1 = Node fastNode2 = startNode;
  while (slowNode && fastNode1 = fastNode2.next() && fastNode2 = fastNode1.next()){
    if (slowNode == fastNode1 || slowNode == fastNode2) return true;
    slowNode = slowNode.next();
  }
  return false;
}

This solution is "Floyd's Cycle-Finding Algorithm" as published in "Non-deterministic Algorithms" by Robert W. Floyd in 1967. It is also called "The Tortoise and the Hare Algorithm".

Robert W. Floyd discoverd the Floyd's cycle-finding algorithm in 1967 based on the properties of the pseudo-random sequences. By applying this algorithm, we simultaneously go through the list by ones (slow iterator) and by twos (fast iterator). If there is a loop the fast iterator will go around that loop twice as fast as the slow iterator. The fast iterator will lap the slow iterator within a single pass through the cycle. Detecting a loop is then just detecting that the slow iterator has been lapped by the fast iterator.

How to remove the loop?

It's obvious now to detect a loop. But how to remove the loop in the linked list? Goal is to locate the place where the loop is occurring and set that node's next to null.

It's actually continuation of the detection process. Once the first two pointers have met, ie., a loop has been detected, freeze the first pointer, and set other pointer pointing to the head, advance these two by 1 step each till the next node to each is the same. The position of the second pointer points to the node that is expected to be the tail, and hence its next should be set to NULL.

But actually it will not work for circlic linked lists, so we need a modification on the algorithm.

Here is the code that I have written for both, does detection and removal. Test code is included, so you can compile and run for testing. I have tested for there cases: No loop case, one loop in the middle case and circlic case, it's OK.

#include 

typedef enum { FALSE, TRUE } bool;

/* Declaration of Node type */
typedef struct node_s {
	int data;
	struct node_s *next;
} node_t;

/*
** Detection: Have two pointers into the list (ptr and depar). Advance depar twice as
** fast as ptr; if they ever point to the same place then the list
** has a loop in it.
**
** Removal: Leave one pointer(depar) in place where the two pointers in the previous
** step met, and another pointer(ptr) pointing to the head, advance these two by 1 step
** each till the next node to each is the same. The node pointer with depar will be the
** last node, so set it's next to NULL.
*/
bool findAndRemoveLoop (node_t *head)
{
	node_t *ptr = head;
	node_t *depar = head;
	bool ret = FALSE;		// the return value
	
	// this loop detects the loop
	while (depar) {
		ptr = ptr->next;
		if ((depar = depar->next) && (depar = depar->next) && depar == ptr) {
			ret = TRUE;
			break;
		}
	}
	
	// this part removes the loop
	if (ret == TRUE) {
		ptr = head;
		while(ptr->next != depar->next) {
			ptr = ptr->next;
			depar = depar->next;
		}
		if(ptr == depar) {	    // note that this means it's a circlic list
			// iterate through the last (prev) node
			while(depar->next != ptr) {
				depar = depar->next;
			}
		}
		depar->next = NULL;	// this points to the last node
	}
	
	return ret;
}


////////////////////// TEST CODE ///////////////////////


node_t nodearr[10];

void initNodes()
{
	int i;
	
	// initialize the nodes
	for(i = 0; i < 10; i++) {
		nodearr[i].data = i;
		nodearr[i].next = NULL;
	}
}

void printList(node_t *head)
{
	int limit = 20;
	while(head && (--limit)) {
		printf("%d->",head->data);
		head = head->next;
	}
	printf("\n");
}

int main()
{
	int i;
	node_t * head;
	
	initNodes();
	
	// establish the linked list
	for(i = 0; i < 9; i++) {
		nodearr[i].next = &nodearr[i+1];
	}
	head = &nodearr[0];
	
	// Scenario 1: No loop
	printf("Scenario 1: No loop\n");
	printList(head);
	printf("Scenario 1: After removal\n");
	findAndRemoveLoop(head);
	printList(head);
	
	// Scenario 2: Loop from node 5	
	// let's have the list loop from node 5
	nodearr[9].next = &nodearr[4];
	printf("Scenario 2: Loop from node 5\n");
	printList(head);
	findAndRemoveLoop(head);
	printf("Scenario 2: After removal\n");
	printList(head);
	
	// Scenario 3: Circlic list
	nodearr[9].next = &nodearr[0];
	printf("Scenario 3: Circlic list\n");
	printList(head);
	findAndRemoveLoop(head);
	printf("Scenario 3: After Removal\n");
	printList(head);	
	
	return 0;
}

Download Code

I hope it helps you. :))

Feed users, click here to go to this topic's page. You can find related topics and reference links and you can read the comments for this topic

Feed users, click here to post a comment for this topic

Note: There might be problems about HTML code and parsing and displaying HTML code of your viewer application. Also, there are problems caused by Google Reader on displaying of code samples on the entry. Also, the entry might be updated. So, it is recommended to visit the page of the entry.

Obfuscated C

Bilal Hatipoglu — 03.02.2008 15:27:17

Obfuscated code is source code that is (usually intentionally) very hard or ambiguous to read and understand with extraneous information. Some languages are more prone to obfuscation than others. C, C++ and Perl are most often cited as easily obfuscatable languages. Macro preprocessors are often used to create hard-to-read code by masking the standard language syntax and grammar from the main body of code.

Code is sometimes obfuscated deliberately for recreational purposes. There are programming contests which reward the most creatively obfuscated code. The contest for C code obfuscation is "The International Obfuscated C Code Contest".

The International Obfuscated C Code Contest (abbr. IOCCC) is a programming contest for the most creatively obfuscated C code held annually since 1984. There are many winning entries each year, and each entry gets a category such as "Worst Abuse of the C preprocessor" or "Most Erratic Behavior".

The IOCCC was started by Landon Curt Noll and Larry Bassel. They were talking together about the horrible code it was their jobs to maintain. They decided to hold a contest for the worst possible C code. Within the code size limit of only a few kilobytes, the contestants manage to do complicated things — a 2004 winner turned out an operating system.

Some ways in which contributions are notable include:

• The appearance of the source code, which may resemble images, text, etc.
• Preprocessor redefinitions to make code harder to read
• Self-modifying code
• Worst abuse of the rules. In several years, an entry was submitted that was so patently absurd that it required a new definition of some of the rules for the next year. This, of course, is a high honor. An example is the world's shortest self-reproducing program. The entry was a program zero bytes in length that if run printed zero bytes to the screen (this requires some creative use of the makefile to get it right). The nature of this contest has naturally resulted in programs which skirt around the edges of C standards, or result in constructs which trigger rarely used code path combinations in compilers. As a result, several of the past entries may not compile directly in a modern compiler, and some may even cause crashes.

An example of obfuscated C code may be this code, which won an award on Obfuscated C Code Contest:

#include 
main(t,_,a)
char *a;
{return!0Download Code

The code above successfully compiles and suprisingly runs and prints an English poem called "The 12 days of christmas":

blaplopl:/mnt/hgfs/bilalshared/deneme # ./siir
On the first day of Christmas my true love gave to me
a partridge in a pear tree.

On the second day of Christmas my true love gave to me
two turtle doves
and a partridge in a pear tree.

On the third day of Christmas my true love gave to me
three french hens, two turtle doves
and a partridge in a pear tree.

On the fourth day of Christmas my true love gave to me
four calling birds, three french hens, two turtle doves
and a partridge in a pear tree.

On the fifth day of Christmas my true love gave to me
five gold rings;
four calling birds, three french hens, two turtle doves
and a partridge in a pear tree.

On the sixth day of Christmas my true love gave to me
six geese a-laying, five gold rings;
four calling birds, three french hens, two turtle doves
and a partridge in a pear tree.

On the seventh day of Christmas my true love gave to me
seven swans a-swimming,
six geese a-laying, five gold rings;
four calling birds, three french hens, two turtle doves
and a partridge in a pear tree.

On the eighth day of Christmas my true love gave to me
eight maids a-milking, seven swans a-swimming,
six geese a-laying, five gold rings;
four calling birds, three french hens, two turtle doves
and a partridge in a pear tree.

On the ninth day of Christmas my true love gave to me
nine ladies dancing, eight maids a-milking, seven swans a-swimming,
six geese a-laying, five gold rings;
four calling birds, three french hens, two turtle doves
and a partridge in a pear tree.

On the tenth day of Christmas my true love gave to me
ten lords a-leaping,
nine ladies dancing, eight maids a-milking, seven swans a-swimming,
six geese a-laying, five gold rings;
four calling birds, three french hens, two turtle doves
and a partridge in a pear tree.

On the eleventh day of Christmas my true love gave to me
eleven pipers piping, ten lords a-leaping,
nine ladies dancing, eight maids a-milking, seven swans a-swimming,
six geese a-laying, five gold rings;
four calling birds, three french hens, two turtle doves
and a partridge in a pear tree.

On the twelfth day of Christmas my true love gave to me
twelve drummers drumming, eleven pipers piping, ten lords a-leaping,
nine ladies dancing, eight maids a-milking, seven swans a-swimming,
six geese a-laying, five gold rings;
four calling birds, three french hens, two turtle doves
and a partridge in a pear tree.

Another interesting award-winning example is the following code:

#include 
#include 
#include 

#define _			;double
#define void			x,x
#define case(break,default)	break[O]:default[O]:
#define switch(bool) 		;for(;xint##if?
#define true			(--void++)
#define false			(++void--)

char*O=" <60>!?\\\n"_ doubIe[010]_ int0,int1 _ Iong=0 _ inIine(int eIse){int
O1O=!O _ l=!O;for(;O1O<010;++O1O)l+=(O1O[doubIe]*pow(eIse,O1O));return l;}int
main(int booI,char*eIse[]){int I=1,x=-*O;if(eIse){for(;I<010+1;I++)I[doubIe-1]
=booI>I?atof(I[eIse]):!O switch(*O)x++)abs(inIine(x))>Iong&&(Iong=abs(inIine(x
)));int1=Iong;main(-*O>>1,0);}else{if(booI<*O>>1){int0=int1;int1=int0-2*Iong/0
[O]switch(5[O]))putchar(x-*O?(int0>=inIine(x)&&do(1,x)do(0,true)do(0,false)
case(2,1)do(1,true)do(0,false)6[O]case(-3,6)do(0,false)6[O]-3[O]:do(1,false)
case(5,4)x?booI?0:6[O]:7[O])+*O:8[O]),x++;main(++booI,0);}}}

Download Code

This small and paranormal code draws the graph of given polynomial on the command line output. Two example runs and outputs are:

blaplopl:/mnt/hgfs/bilalshared/deneme/2004 # ./hoyle 0 0 1
\_                              |                              A
  \                             |                             /
   \                            |                            /
    \                           |                           /
     \                          |                          /
      \                         |                         /
       \                        |                        /
        \_                      |                      A/
          \                     |                     /
           \_                   |                   A/
             \_                 |                 A/
               \                |                /
                \__             |             A_/
                   \_           |           A/
                     \__        |        A_/
                        \_______|A______/
________________________________V_______________________________
                                |
                                |
                                |
                                |
                                |
                                |
                                |
                                |
                                |
                                |
                                |
                                |
                                |
                                |
                                |
blaplopl:/mnt/hgfs/bilalshared/deneme/2004 # ./hoyle 0 0 1 2
                                |                              /
                                |
                                |                             /
                                |                            /
                                |                           /
                                |
                                |                          /
                                |                         /
                                |                        /
                                |                      A/
                                |                     /
                                |                   A/
                                |                  /
                                |               A_/
                                |            A_/
                                |A__________/
____________________A___________/_______________________________
                A__/            |
              A/                |
            A/                  |
           /                    |
         A/                     |
        /                       |
       /                        |
      /                         |
     /                          |
    /                           |
   /                            |
                                |
  /                             |
 /                              |
                                |

It's working, isn't it?

Another famous C example is:

_(__,___,____){___/__<=1?_(__,___+1,____):!(___%__)?_(__,___+1,0):___%__==___/
__&&!____?(printf("%d\t",___/__),_(__,___+1,0)):___%__>1&&___%__<___/__?_(__,1+
___,____+!(___/__%(___%__))):___<__*__?_(__,___+1,____):0;}main(){_(100,0,0);}

This program prints out the prime numbers less than 100. While this program is extremely difficult to follow, it is the result of several simple transformations from the following elementary program:

void primes(int cap) {
  int i, j, composite;
  for(i = 2; i < cap; i++) {
    composite = 0;
    for(j = 2; j < i; j++) 
      composite += !(i % j);
    if(!composite)
      printf("%d\t", i);
  }
}
 
int main() { 
  primes(100);
}

The first set of transformations aim to reduce the primes function into a single statement: (1) the meat of the function is rewritten as a single while loop with a sequence of cascading if-else structures inside of it; (2) the while loop is then rewritten as recursion; (3) the cascading if-else statements is rewritten as a single conditional statement. These transformations serve to obfuscate the code considerably. The remaining transformations purposefully hide the details of the function by: (4) removing all intermediate variables; (5) renaming the few variables left to _, ___, etc.; and, finally, (6) removing all whitespace and unnecessary parentheses.

The first transformation takes advantage of the well-known property that any simple function can be transformed into a single while loop followed by a series of cascading if-else statements. The program above is first transformed by rewriting the primes function in this form:

void primes(int cap) { 
  int i, j, composite, t = 0;
  while(t < cap * cap) {
    i = t / cap;
    j = t++ % cap;
    if(i <= 1);
    else if(j == 0)
      composite = 0;
    else if(j == i && !composite)
      printf("%d\t",i);
    else if(j > 1 && j < i)
      composite += !(i % j);  
  }
}
 
int main() {
  primes(100);
}

Another well-known property is that any while loop can be replaced by a series of recursive calls. The program is further transformed by replacing the while loop with recursion. Note that this requires adding two parameters to the primes function's header. Also note the consolidation of two statements into a list in the third if block. Finally note that one additional if-else block must be added to capture the situation in the non-recursive version where program flow fails all of the if-conditions and the while loop would simply continue with t incremented by one:

void primes(int cap, int t, int composite) {
  int i,j;
  i = t / cap;
  j = t % cap;
  if(i <= 1)
    primes(cap,t+1,composite);
  else if(j == 0)
    primes(cap,t+1,0);
  else if(j == i && !composite)
    (printf("%d\t",i), primes(cap,t+1,0));
  else if(j > 1 && j < i)
    primes(cap,t+1, composite + !(i % j));
  else if(t < cap * cap)
    primes(cap,t+1,composite);
}
 
int main() {
  primes(100,0,0);
}

The program is next transformed by changing the variable names to single letters and replacing the if-else structure with conditional statements (e.g. if(A) B else if(C) D else E becomes A ? B : C ? D : E):

void primes(int m, int t, int c) {
  int i,j;
  i = t / m;
  j = t % m;
  (i <= 1) ? primes(m,t+1,c) : (j == 0) ? primes(m,t+1,0) : (j == i && !c) ? 
  (printf("%d\t",i), primes(m,t+1,0)) : (j > 1 && j < i) ? 
  primes(m,t+1,c + !(i % j)) : (t < m * m) ? primes(m,t+1,c) : 0;
}
 
int main() {
  primes(100,0,0);
}

Next, the program is transformed by removing the intermediate variables i and j and replacing them with (t / m) and (t % m), respectively:

void primes(int m, int t, int c) {
  ((t / m) <= 1) ? primes(m,t+1,c) : !(t % m) ? primes(m,t+1,0) : 
  ((t % m)==(t / m) && !c) ? (printf("%d\t",(t / m)), primes(m,t+1,0)) : 
  ((t % m)> 1 && (t % m) < (t / m)) ? primes(m,t+1,c + !((t / m) % (t % m))) : 
  (t < m * m) ? primes(m,t+1,c) : 0;
}
 
int main() {
  primes(100,0,0); 
}

Next, the program is transformed by renaming the function primes and the variables m, t, and c to _, __, ___, and ____, respectively:

void _(int __, int ___, int ____) {
  ((___ / __) <= 1) ? _(__,___+1,____) : !(___ % __) ? _(__,___+1,0) : 
  ((___ % __)==(___ / __) && !____) ? (printf("%d\t",(___ / __)), 
  _(__,___+1,0)) : ((___ % __) > 1 && (___ % __) < (___ / __)) ? 
  _(__,___+1,____ + !((___ / __) % (___ % __))) : (___ < __ * __) ? 
  _(__,___+1,____) : 0;
} 
 
int main() {
  _(100,0,0); 
}

Finally, the program is transformed by removing white space, type declarations, and unambiguous parentheses to come up with the fully obfuscated version:

_(__,___,____){___/__<=1?_(__,___+1,____):!(___%__)?_(__,___+1,0):___%__==___/
__&&!____?(printf("%d\t",___/__),_(__,___+1,0)):___%__>1&&___%__<___/__?_(__,1+
___,____+!(___/__%(___%__))):___<__*__?_(__,___+1,____):0;}main(){_(100,0,0);}

This program runs on most systems. The limiting factor generally will be stack overflow because setting the equivalent of the cap parameter to 100 generally results in 100^2 recursive calls. This sort of obfuscation by program transformation is relatively easy to apply and can be performed on many simple programs.

There are many varieties of interesting obfuscations ranging from simple keyword substitution, use/non-use of whitespace to create artistic effects, clever self-generating or heavily compressed programs, or programs that are valid and operate similarly in multiple programming languages.

You can find really very compilated and interesting, award winning codes an programs on "The International Obfuscated C Code Contest Winners by Years". There are really interesting ones for example a 32-bit multitasking operating system, a flight simulation program, a maze game, a poker game etc.

Note: You can download and try the codes below. I have tested each on Linux environment and compiled with GCC. The codes may get unexpected warnings, may not be compiled or run depending on your environment and compiler.

Feed users, click here to go to this topic's page. You can find related topics and reference links and you can read the comments for this topic

Feed users, click here to post a comment for this topic

Note: There might be problems about HTML code and parsing and displaying HTML code of your viewer application. Also, there are problems caused by Google Reader on displaying of code samples on the entry. Also, the entry might be updated. So, it is recommended to visit the page of the entry.

Is C a Vitamin? Yes, of course...

Bilal Hatipoglu — 01.02.2008 08:05:54

This article is actually a followup for the entry of Ozgur Macit, on his blog called 'Is C a vitamin? He criticies the missing features of C, importance of C and the teaching of C for Engineering students.

Please go and read the referred article before this entry.

At first, let me congratulate Ozgur Macit for this professional entry. it's good to have references, even for a single blog entry, this gives a professionality and article like dignity. But this negative critisism bites me :) , I think there are many worng decisions and iniquity for C, so I've considered to follow up.

1) Readability (yes, it is Read'A'bility, not Readibility)

The deep mistake on the entry is, you are comparing C, the low-end of high level languages(this original definition is written by me :) ) with a very-high language, Python. They are so irrelevant that, they are for different purposes. This is from one of your references, "Comparison of programming languages", on wikipedia:

"Language: C
Intended Use: System
Design Goal: Low level access, Minimal constraint

Language: Python
Intended Use: Application, Educational, Scripting
Design Goal: Simplicity, Readability, Expressiveness, Modularity"

Although this article an wikipedia has many missing things, are there any similarity? No, right. Let's exploit wikipedia, for the purpose and deign goal of C:

"C is a general-purpose, block structured, procedural, imperative computer programming language developed in 1972 by Dennis Ritchie at the Bell Telephone Laboratories for use with the Unix operating system. It has since spread to many other platforms. Although C was designed as a system implementation language, it is also widely used for applications. C has also greatly influenced many other popular languages, especially C++, which was originally designed as an extension to C.

C is an imperative (procedural) systems implementation language. Its design goals were for it to be compiled using a relatively straightforward compiler, provide low-level access to memory, provide language constructs that map efficiently to machine instructions, and require minimal run-time support. C was therefore useful for many applications that had formerly been coded in assembly language.

Despite its low-level capabilities, the language was designed to encourage machine-independent programming. A standards-compliant and portably written C program can be compiled for a very wide variety of computer platforms and operating systems with minimal change to its source code. The language has become available on a very wide range of platforms, from embedded microcontrollers to supercomputers..."

Let me summarize the main goal of the C, the main purpose of use of C: Writing memory and CPU efficient programs, relatively easy to compile(fast compilation), easy to code near the level of assembly languages. It's so low-level that, you can imagine the assembly equivalent of the C code while you are writing the C code :)

I'm not so familiar with Python, but I know it's for a different purpose, with different pros and cons, with different(higher) level of programming.

If you want to compare C with another language, this must be Pascal, or Basic or whatever, but with the similar level, with the similar efficiency and similar purpose. You will see it's mostly Pascal, if you want to compare C with a language.

Now, go and see the comparison on wikipedia: Comparison of Pascal and C (http://en.wikipedia.org/wiki/Pascal_and_C)

Then, you may not say anything about C's readability. Readability is the primary benefit for whom uses C. You may not know Pascal, but you will see it's very similar to PL/SQL in the syntax. You have the begin-end blocks, if-then-end if, etc. It's very hard to read such a code, compared to the C equivalent, isn't it?

Also, from a simple Google search, you can find the links:

• Top 10 Reasons Pascal beats C
• C vs. Pascal
• Why Pascal is Not My Favorite Programming Language

If you read them, you can see it's also very ridiculous to compare Pascal and C, because no way to have a better comment than:

"The question asked is: Which is better or more suited to a particular task ?

The answer ? There is no answer !"

And also I love the moral of the story:

"Bottom line is:

Choose the tool you are at ease with and it will serve you well. The C vs Pascal debate is a waste of time. "

(Both quoted from C vs. Pascal)

And maybe you will have a chance to understand how funny to compare C and Python, or Java or something like.

2) Orthogonality

The example ypu have given is not so strange, you can do the same thing with MALLOC kind of stuff, you know. You have addressed the problem of C doesn't have a native String type. That's what the higher-level languages like Java or Python do for storing a String: Dynamic allocation! Since the language does it automatically, it comes so simple for the programmer. You have the simplicity, but you lost the conrol, you lost the efficiency.

It's not he Orthogonality issue as you said:

"Programmer occasionally does not have chance to write a similar code for two similar instances of same problem in C. For this reason, C is said to be not orthogonal."

And let me write the similar code for the same problem that you addressed. You can compile and run, it's working:

#include 
#include 
#include 


char * double_str(char * s) {
	int len = strlen(s) * 2 + 1;
	char *t = malloc(sizeof(char)*len);
	
	strcpy(t, s);
	strcat(t, s);
	
	t[len-1] = '\0';
	
	return t;
}

int main() {
	char s[] = "hello bilal!";
	
	char *ret = double_str(s);
	
	printf("%s\n",ret);
	
	return 1;
}

3) Safety

• "Though type checking is done by compiler, programmer is free to make type casting."

Yes, that what we "the C coders" love. What's wrong with it? In C, the programmer have the power but he also have the responsibility.

• "C never and never makes index range checking. It is claimed that C encourages buffer overflows with this property in [2]. Same source gives a list of functions that may cause buffer overflow accidentally."

As I said, "In C, the programmer have the power but he also have the responsibility". Index range checking is not the problem of C, it's actually the problem of the compiler. I know some C compilers does index range checking, but actually it's disabled feature traded to have efficiency.

Consider a huge loop:

char *c;

...


for(i = 0; i < 999; i++) {
	c[i] = i;
}

And the "sweety"(!) compiler adds the index range checking code:

char *c;

...


for(i = 0; i < 999; i++) {
	if(i > index_range_of_c) {	// whatever, the compiler knows
		printf("Index range out of bounds: c[%d]\n",i);
		return;	// or exit();
	}
	c[i] = i;
}

and compare the execution time (and also compilation effort, not for this case but for general case) of each. Than come and say "Why don't C have index range checking? C is an evil."

• "Programmer has the memory! No safety check is done about memory allocation."

Yes. "In C, the programmer have the power but he also have the ..." :)

C let's programmer to write some code, not writes the code, generates the algorithm and does the job of programmer etc. like Java or Python.

• "You must not free same pointer twice - again accidentally, if you do you are in trouble. There is no internal mechanisms to avoid this. Programmer must always check the pointer while allocating and freeing memory."

If programmer needs to check it, he checks it. If the programmer knows what he is doing, no problem. For compiler, why bother and trade the performance for collecting the "garbage" of the programmer every time, so pump and exaggerate the code which will steal my machine cycles and memory?

4) Redundancy

"But like a lot of similar things it cannot be fully thrown out of language, because backward compatibility is needed."

Go and look the documentation of Java. You will see a lot of stuff that is deprecated, although, Java is a brand-new language (developed in mid 90's). C is an old language, yes, some functions have security problems or bugs, but it's normal and not so much.

“In many ways, the C language evolved into a collection of overlapping features...”

Overlapping features?? Yes C has overlapping features, but which "old" language doesn't have? The brand-new Java although have much more overlapping features and non-reliabilities. Go and look about it.

“... providing too many ways to say the same thing, while in many cases not providing needed features.”

C not providing needed features?? Oh no, why don't you find a psychiatrist, who can carefully listen and confirm you and what you are saying about C?

Yes a "White Paper" of Java can say that. I had always encouraged them to go and look about inside Java. :))

5) String type, Error Handling

Yes, these are considered as missing features also in C community. But one can do what he want with writing some more code and with some workaround, right?

6) Reaching Elements

"Another complaint about C is why do we have both . and -> to use for the same purpose. Firstly, they are not for the same purpose. '.' is structure offset and -> is used for dereferencing. Sure we should expect from compiler to take care of this difference or we can simply do not want to help the compiler to ease its job."

Why not? Is the compiler your enemy, or opponent? It seems to be a simple feature for compiler, but it's actually not easy to keep track of. Notice one of the main purposes of C: Easy, fast compilation. C is designed as it can be compiled also on limited environments.

"But in my opinion understandability of code increases with this small difference. Looking at the code we can easily see what is a pointer and we are dereferencing its indigrents and what is a name of a real structure. Let me say I never liked Java way of avoiding pointers."

I can't understand on which side are you but with two different identifiers '.' and '->' one can easily understand if it is a pointer or structure itself. It's more readable. Pascal has also just the '.', but it's just lack of readability I think.

7) goto statement

"James A. C. Joyce claims that using a goto statement is the only way of breaking out of nested for or while loops in [2]."

Yes, he now compared with PHP!! What about Pascal or Basic? And he says:

"A couple of examples should illustrate what I mean quite nicely. If you've ever programmed in PHP for a substantial period of time, you're probably aware of the 'break' keyword. You can use it to break out from nested loops of arbitrary depth by using it with an integer, such as "break 3"; this would break out of three levels of loops.

There is no way of doing this in C. If you want to break out from a series of nested for or while loops then you have to use a goto. This is what is known as a crude hack."

Yes, we appreciate Kernighan for providing us this hack :))

"In addition to this, there is no way to compare any non-numerical data type using a switch statement. C does not allow you to use switch and case statements for strings. One must use several variables to iterate through an array of case strings and compare them to the given string with strcmp(). This reduces performance and is just yet another hack.

In fact, this is an example of gratuitous library functions running wild once again. Even comparing one string to another requires use of the strcmp() function."

Dude, do you know how the "high-level languages" that you glorify does that stuff of 'String comparison'?? It's just the same of strcmp() at the lower-level, it must be, the compiler automatically does it, with even worse performance than C. Yes, in C, the "switch" statement does not allow strings, but even the "case of" statement in Pascal not. The features of brand-new, higher level languages fascinate you, but you even don't know what they are stealing from your brand-new GHZ'es of CPU's, GB's of RAM, and high-end hardware configuration.

I don't know who James 'whatever' is, but he seems to be a good candidate for being 'The Burlescon of C' :))

Note: Yes, goto statement lacks of readability, but nobody encourages to use it for most of the languages. It's not an issue for just C.

8) Enums

"C’s enum structure has a very significant and important problem which may be easily solved with object-oriented programming. If you use a name in an enum, you cannot use this name in another enum. If you use an object-oriented programming language like Java, you may put the same constant name inside different classes."

Yes, it seems from now on you are comparing C with Java. :)) Don't make me laugh... If you want OO stuff (with lot's of overhead of inefficient execution) go and use Java, Python or whatever, but don't say anything to C, it's just a structural language. Why don't you compare the OO features of 'your Python' and Java?

9) What C doesn’t have

"• Exception handling mechanism "
"• Function overloading "

Yes it doesn't. Nothing to say.

"• Specialized data types"

Object-Oriented features?? :-s

"• Garbage collection"

Consider the garbage of who? :)) C executes the code that it is responsible to, while the 'Glorious Garbage Collector" collects 'The garbage of the programmer' in other languages :)) Consider the overhead of the garbage collector, I will write a separate article about it later. I can't belive you are talking about a lacking "garbage collector" under "Waht C doesn't have" :-s

The Conclusion

"C is not the evil in this story, I think. Just, it is not really proper for high-level programming. It must be used for what it is designed for.

Some bloody properties cannot be abandoned due to backward compatibility. Newbies always discouraged using these features by experienced programmers."

Let them be discouraged about it. They will be discouraged many times in the future. It's the reality of life :)) Notice I was discouraged when I have to create a huge "Integer" object for just storing a 4-byte integer value, and also lack of freeing that object from memory when I was done with that object, when I was learning Java.

"While coding in C, I feel myself in the middle of 70s while memory was so important and that I shouldn’t use a single byte if I really don’t need it."

Yes, sometimes you need to feel yourself like that. A good programmer always has to feel that. The whole CPU and GB's of cheap memory is not the right thing you may lean on. :)) Never!

Consider a Systems Programmer or Embedded Environment. That's why it's hard to do that stuff. That's why people can easily learn and develop applications in Java or C#, but not easily learn C and develop the deeper ones in C. I think that must be the difference of a person who got a 4-year Computer Engineering education.

"This makes me sick about C :)"

And this makes me love C :))

As an overall conclusion for your claim about "Is C the best language to teach as the first programming language, for Engineering students":

If you first teach students Java instead of C, you can never able to:

1) Teach them the C,
2) The importance and the purpose of C,
3) The importance of efficient programming,
4) The Systems Programming Concept,
5) The Embedded Programming Concept,
6) Even the assembly language :) (Already they are not able to learn assembly, even they are first learning C or other high-level languages)
7) The basic concept and principles of programming languages. Ex: You cannot teach what Java does when ou create an object with New(), if one doesn't already know about what MALLOC() does.

Since the aim of Computer Engineering study is the stuff above, It is mandatory to teach C, not only teaching the syntax of C, but also teaching the fundamentals of C and teaching them well better than other courses is mandatory.

But everybody who learns or who knows about Java or other higher level languages starts to criticise on C or the study of C. Sorry, but it's a bullshit.

"Is C a vitamin?" Yes, of course. Take one-a-day :)) or many, when you need, when you can't solve -as the majority of- real engineering problems(like Systems Programming, Embedded Programming, Signal-Image-Voice-Speech Processing, and similar things requires design of efficient applications and algorithms) with higher level languages.

Feed users, click here to go to this topic's page. You can find related topics and reference links and you can read the comments for this topic

Feed users, click here to post a comment for this topic

Note: There might be problems about HTML code and parsing and displaying HTML code of your viewer application. Also, there are problems caused by Google Reader on displaying of code samples on the entry. Also, the entry might be updated. So, it is recommended to visit the page of the entry.

Differences between C and C++

Bilal Hatipoglu — 29.01.2008 23:58:06

The C programming language was designed by Dennies Ritchie in the early 1970s at Bell Laboratories. It was first used system implementation language for the nascent Unix operating system. The main reason to devised C was to overcome the limitations of B. It was Derived from the type-less language BCPL ((Basic Combined Programming Language). C was was the evolution of B and BCPL by incorporating type checking. It was originally intended for use in writing compilers for other languages.

C++ was devised by Bjarne Stroustrup in 1983 at Bell Laboratories. It is an extension of C by adding some enhancements to C language. Bjarne combined the simula's(a language designed for making simulations, created by Ole-Johan Dahl and Kristen Nygaard) features of object oriented and the efficiency of C. The new features added to language are templates, namespaces, exception handling and use of standary library.

C and C++ are no more language for writing compilers and other languages, these general purpose languages are used worldwide in every field.

C was the C++ predecessor. As it's name implies, a lot of C remains in C++. Although not actually being more powerful than C, C++ allows the programmer to more easily manage and operate with Objects, using an OOP (Object Oriented Programming) concept.

The main difference between C and C++ is that C++ is object oriented while C is function or procedure oriented. Object oriented programming paradigm is focused on writing programs that are more readable and maintainable. It also helps the reuse of code by packaging a group of similar objects or using the concept of component programming model. It helps thinking in a logical way by using the concept of real world concepts of objects, inheritance and polymorphism. It should be noted that there are also some drawbacks of such features. For example using polymorphism in a program can slow down the performance of that program.

On the other hand, functional and procedural programming focus primarily on the actions and events, and the programming model focuses on the logical assertions that trigger execution of program code.

C++ allows the programmer to create classes, which are somewhat similar to C structures. However, to a class can be assigned methods, functions associated to it, of various prototypes, which can access and operate within the class, somewhat like C functions often operate on a supplied handler pointer.

Although it is possible to implement anything which C++ could implement in C, C++ aids to standarize a way in which objects are created and managed, whereas the C programmer who implements the same system has a lot of liberty on how to actually implement the internals, and style among programmers will vary alot on the design choices made.

In C, some will prefer the handler-type, where a main function initializes a handler, and that handler can be supplied to other functions of the library as an object to operate on/through. Others will even want to have that handler link all the related function pointers within it which then must be called using a convention closer to C++.

To finish this discussion, C++ applications are generally slower at runtime, and are much slower to compile than C programs. The low-level infrastructure for C++ binary execution is also larger. For these reasons C is always commonly used even if C++ has a lot of popularity, and will probably continue to be used in projects where size and speed are primary concerns, and portable code still required (assembly would be unsuitable then).

Another difference is that, type checking for example is much more rigid in C++ than it is in C, so many a program that compiles just fine under a C compiler will result in many warnings and errors under a C++ compiler.

So, while C++ might be seen as "C with classes" by some, it actually is different and targeted at a different audience. C still is the best choice for code that has to be fast while still being reasonably readable and portable, eg. device drivers. C++ on the other hand is mainly used in large projects of millions of lines of code, where C code would become unmaintainable. This is mainly due to the possibility to reuse existing code by exploiting the OOP concepts of polymorphism and inheritance.

C++ Features Not in C

Flexible Declarations

In C, all var declarations within a scope occur at the beginning of that scope. Thus, all global declartions must appear before any functions, and any local declarations must be made before any executable statements. C++, on the other hand, allows you to mix data declarations with functions and executable statements. E.g. In C,

         void makeit(void)
         {
            float i;
            char *cp;

            /* imagine 2000 lines of code here */

            /* allocate 100 bytes for cp */
            cp = malloc(100);    /* 1st use of cp */

            for (i=0; i<100; ++i)      /* 1st use of i */
            {
               /* do something */
            }

            /* more code */
         }

In C++,

         void makeit(void)
         {
            // 2000 lines of code

            char *cp = new char[100];

            for (int i=1; i<10; i++)
            {
            }
         }

Struct, Union and Enum Tags

In C, we would have this segment:

         struct foo {int a; float b;}
         struct foo f;

This declares a struct with the tag name 'foo' and then creates an instance of foo named f. Notice when you declare var of that struct, you have to say 'struct foo'. In C++, struct and union tags are considered to be type name, just as if they had been declared by the 'typedef' statement.

         struct foo {int a; float b;}
         foo f;

which is equivalent to the following in C:

         typedef struct
                 {
                     int a;
                     float b;
                 } foo;
         foo f;

You have to include the struct keyword before the name of the struct type to declare a struct: In C++, you could do this

	struct a_struct
	{
		int x;
	};
	
	a_struct struct_instance;

and have a new instance of a_struct called struct_instance. In C, however, we have to include the struct keyword when declaring

	struct_instance: 
	struct a_struct struct_instance;

In fact, a similar situation also holds for declaring enums: in C, you must include the keyword enum; in C++, you don't have to. As a side note, most C programmers get around this issue by using typedefs:

	typedef struct struct_name
	{
		/* variables */
	} struct_name_t;

Now you can declare a struct with

	struct_name_t struct_name_t_instance;

But there is another gotcha for C++ programmers: you must still use the "struct struct_name" syntax to declare a struct member that is a a pointer to the struct.

	typedef struct struct_name
	{
		struct struct_name instance;
		struct_name_t instance2; /* invalid!  The typedef isn't defined yet */
	} struct_name_t;

Same thing is appliciable for Enum and Union.

Const

In ANSI C, it also supports 'const', but C++'s 'const' is more flexible than C's. In both C and C++, a value declared as 'const' is inviolate; it may not be modified by any part of the program in any way. The most common use of 'const' values in C is to replace '#define' literal constants.

         #define MAX_CUSTOMERS   10
         const int MAX_CUSTOMERS = 10;

Thus,

         MAX_CUSTOMERS = 10;
         MAX_CUSTOMERS ++;

are both not acceptable. Note: since you cannot make changes to a 'const', each constant must be initialized when declared. The following is wrong:

         const int invalid;

In C++, you can do something like

         const int ArraySize = 100;
         int Array[ArraySize];

while in ANSI C, this would be flagged as an error.

More examples for 'const':

         const int v[] = {1, 2, 3, 4};

         const char* pc = "asdf";   // pointer to constant
         pc[3] = 'a';               // error
         pc = "ghjk";               // ok

         char *const cp = "asdf";   // constant pointer
         cp[3] = 'a';               // ok
         cp = "ghjk";               // error

         const char *const cpc = "asdf";  // const pointer to const
         cpc[3] = 'a';                    // error
         cpc = "ghjk";                    // error

Function call:

         char* strcpy (char* p, const char*q);  // cannot modify *q

Here, by declaring a pointer argument const, the function is prohibited from modifying the object pointed to, which makes perfect sense since strcpy copies q to p and you better preserve q.

The :: Operator

:: is called the scope resolution operator and is used to access an item hidden in the current scope. For example,

         int a;

         main()
         {
            float a;

            a = 1.5;
            ::a = 2;

            cout << "local a=" << a << endl;
            cout << "global a=" << ::a << endl;

The :: operator says, "don't use the local a, use the one declared outside the scope". Thus:

         local a=1.5
         global a=2

It is to be noted, however, that it's possible to use the global var without the :: operator:

         int x = 11;

         void f4()
         {
            int y = x;     // global x
            int x = 22;
            y = x;         // local x
         }

New and Delete

In C, all dynamic mem allocation is handled via library calls, such as 'malloc' and 'free'. Here's how a traditional C programs might allocate memory:

         void func(void)
         {
            int *i;

            i = (int *)malloc(sizeof(int));
            *i = 10;
            printf("%d", *i);
            free(i);
         }

In C++, there are new ways of dynamically allocating mem using operators called 'new' and 'delete', where 'new' replaces 'malloc' and 'delete' replaces 'free' in C. We could rewrite the above function as the following:

         void func()
         {
            int *i = new int;

            *i = 10;
            cout << *i;
            delete i;
         }

You'd probably agree this is a much clearer syntax and it's much easier to use as well. A couple more examples:

         int *i = new int[10];    //  an array of 10 integers
         int *i[10] = new int(*)[10]; // an array of 10 pointers to integers

You can also intialize all the variables allocated by 'new':

         float *f = new float[50] (0.0);

In C, there's only one major memory allocation function: malloc. You use it to allocate both single elements and arrays:

	int *x = malloc( sizeof(int) );
	int *x_array = malloc( sizeof(int) * 10 );

and you always release the memory in the same way:

	free( x );
	free( x_array );

In C++, however, memory allocation for arrays is somewhat different than for single objects; you use the new[] operator, and you must match calls to new[] with calls to delete[] (rather than to delete).

	int *x = new int;
	int *x_array = new int[10];
	
	delete x;
	delete[] x;

The short explanation is that when you have arrays of objects, delete[] with properly call the destructor for each element of the array, whereas delete will not.

Note: Don't mix the use of 'new' and 'delete' with that of 'malloc' and 'free'. i.e, always use either all the C lib calls 'malloc' and 'free' in your program to manage dynamic mem. OR use all 'new' and 'delete'. All the mem allocated by 'malloc' should be returned to the available mem pool by 'free' and the same holds for 'new' and 'delete'. I'd just use 'new' and 'delete'.

References

C can by clumsy sometimes. When you write a function to swap two integers, you have to pass the two integers into the function by reference:

         void swapint(int *a, int *b)
         {
            int temp;

            temp = *a;
            *a = *b;
            *b = temp;
         }

Here's the function call:

         swapint(&i1, &i2);

C++ supports a special type of identifier known as 'reference' &. It makes changing the parameter values in a function reletively painless. The above function can be rewritten in C++ as follows:

         void swapint(int &a, int &b)
         {
            int temp = a;
            a = b;
            b = temp;
         }

Function call:

         swapint(i1, i2);

When i1 and i2 are passed into the function, a POINTS to i1 and b POINTS to i2 instead of making local copies of i1 and i2. Now, whenever you refer to a or b in the function, you actually refer to i1 and i2. So, whatever changes you make to a or b, they will be reflected on i1 and i2.

Function Overloading

In C, as in most other programming languages, every function must have a unique name. At times, it can be annoying. Imagine you want to have a function that returns the abs value of an integer.

         int abs(int i);

If you need to figure out the abs value of every possible available data type, you then have to write a function for each of the possible types:

         long labs(long l);
         double dabs(double d);

All those functions do the same thing -- return the abs value of the argument. Thus it seems silly to have a different name for each of those functions. C++ solves this by allowing you to create those functions with the same name. This is called overloading. For example, you can do the above in C++:

         int abs(int i);
         long abs(long l);
         double abs(double d);

And depending on the type of parameter you pass into the 'abs' func. C++ will select the right one. Note: What if the type of the parameter passed in is not identical to any of the available parameter types in the existing functions?

               abs('a');
               abs(3.1415F);

C++ will try to make the easiest conversion to match those parameter types in the funcion prototypes.

               abs('a');      // call int abs(int i)
               abs(3.1415F);  // call double abs(double d);

If no such conversion exists, then an error will occur.

There is a huge article (as large as to be referred as a book :) ) in here, which covers Incompatibilities and many differences Between ISO C and ISO C++, but I only covered some important ones.

For the last word, there are many features in C++ which are not exist in C, but many people already prefers C for the reasons like: the syntax is much easier to write in C, harder to compile C++ code (not harder for user but harder for computer :) ), and for whom who cares about a single byte to save from Memory or a single cycle to save from CPU, C is more efficient :) PS: I am lovin' it ;)

Feed users, click here to go to this topic's page. You can find related topics and reference links and you can read the comments for this topic

Feed users, click here to post a comment for this topic

Note: There might be problems about HTML code and parsing and displaying HTML code of your viewer application. Also, there are problems caused by Google Reader on displaying of code samples on the entry. Also, the entry might be updated. So, it is recommended to visit the page of the entry.

Whence C? Why C? Whither C?

Bilal Hatipoglu — 29.01.2008 23:52:34

As about ten years ago, C was the most popular programming language being used. Programmers wrote C code for PCs, mainframes, and supercomputers. C was standardized so that the same program would run on every kind of computer using every kind of operating system available.

Today, the use of C has dramatically decreased. Rarely will a project be written in C. Programmers are using languages such as C++ and Java to create applications that run on standalone, networked, and Internet-based machines. Yet, when learning to program, C is considered to be the one language it is imperative a programmer master before moving on to the other languages in use today.

The reason for C's recommended mastery, despite its low levels of actual use, is that C is the common denominator of many of today's languages; learn C and these languages will be simple to learn. Languages such as C++ and Java are based on C. In fact, many of the statements and commands in these newer languages are identical to the ones found in C. Overall, C is much simpler to learn than these languages because it carries with it a much slimmer toolkit of add-on procedures.

The article below is taken from the first chapter of the book "C Primer Plus, Fifth Edition, By Stephen Prata, 2004".

Whence C?

Dennis Ritchie of Bell Labs created C in 1972 as he and Ken Thompson worked on designing the Unix operating system. C didn't spring full-grown from Ritchie's head, however. It came from Thompson's B language, which came from… but that's another story. The important point is that C was created as a tool for working programmers, so its chief goal is to be a useful language.

Most languages aim to be useful, but they often have other concerns. The main goal for Pascal, for instance, was to provide a sound basis for teaching good programming principles. BASIC, on the other hand, was developed to resemble English so that it could be learned easily by students unfamiliar with computers. These are important goals, but they are not always compatible with pragmatic, workaday usefulness. C's development as a language designed for programmers, however, has made it one of the modern-day languages of choice.

Why C?

During the past three decades, C has become one of the most important and popular programming languages. It has grown because people try it and like it. In the past decade, many have moved from C to the more ambitious C++ language, but C is still an important language in its own right, as well a migration path to C++. As you learn C, you will recognize its many virtues (see Figure 1.1). Let's preview a few of them now.

Figure 1.1. The virtues of C.

Design Features

C is a good language incorporating the control features found desirable by the theory and practice of computer science. Its design makes it natural for top-down planning, structured programming, and modular design. The result is a more reliable, understandable program.

Efficiency

C is an efficient language. Its design takes advantage of the capabilities of current computers. C programs tend to be compact and to run quickly. In fact, C exhibits some of the fine control usually associated with an assembly language. (An assembly language is a mnemonic representation of the set of internal instructions used by a particular central processing unit design; different CPU families have different assembly languages.) If you choose, you can fine-tune your programs for maximum speed or most efficient use of memory.

Portability

C is a portable language, which means that C programs written on one system can be run on other systems with little or no modification. If modifications are necessary, they can often be made by simply changing a few entries in a header file accompanying the main program. Most languages are meant to be portable, but anyone who has converted an IBM PC BASIC program to Apple BASIC (and they are close cousins) or has tried to run an IBM mainframe FORTRAN program on a Unix system knows that porting is troublesome at best. C is a leader in portability. C compilers (programs that convert your C code into the instructions a computer uses internally) are available for about 40 systems, running from 8-bit microprocessors to Cray supercomputers. Note, however, that the portions of a program written specifically to access particular hardware devices, such as a display monitor, or special features of an operating system, such as Windows XP or OS X, typically are not portable.

Because of C's close ties with Unix, Unix systems typically come with a C compiler as part of the packages. Linux installations also usually include a C compiler. Several C compilers are available for personal computers, including PCs running various versions of Windows, and Macintoshes. So whether you are using a home computer, a professional workstation, or a mainframe, the chances are good that you can get a C compiler for your particular system.

Power and Flexibility

C is powerful and flexible (two favorite words in computer literature). For example, most of the powerful, flexible Unix operating system is written in C. Many compilers and interpreters for other languages—such as FORTRAN, Perl, Python, Pascal, LISP, Logo, and BASIC—have been written in C. As a result, when you use FORTRAN on a Unix machine, ultimately a C program has done the work of producing the final executable program. C programs have been used for solving physics and engineering problems and even for animating special effects for movies such as Gladiator.

Programmer Oriented

C is oriented to fulfill the needs of programmers. It gives you access to hardware, and it enables you to manipulate individual bits in memory. It has a rich selection of operators that allows you to express yourself succinctly. C is less strict than, say, Pascal in limiting what you can do. This flexibility is both an advantage and a danger. The advantage is that many tasks, such as converting forms of data, are much simpler in C. The danger is that with C, you can make mistakes that are impossible in some languages. C gives you more freedom, but it also puts more responsibility on you.

Also, most C implementations have a large library of useful C functions. These functions deal with many needs that a programmer commonly faces.

Shortcomings

C does have some faults. Often, as with people, faults and virtues are opposite sides of the same feature. For example, we've mentioned that C's freedom of expression also requires added responsibility. C's use of pointers (something you can look forward to learning about in this book), in particular, means that you can make programming errors that are very difficult to trace. As one computer preliterate once commented, the price of liberty is eternal vigilance.

C's conciseness combined with its wealth of operators make it possible to prepare code that is extremely difficult to follow. You aren't compelled to write obscure code, but the opportunity is there. After all, what other language has a yearly Obfuscated Code contest?

There are more virtues and, undoubtedly, a few more faults. Rather than delve further into the matter, let's move on to a new topic.

Whither C?

By the early 1980s, C was already a dominant language in the minicomputer world of Unix systems. Since then, it has spread to personal computers (microcomputers) and to mainframes (the big guys). See Figure 1.2. Many software houses use C as the preferred language for producing word processing programs, spreadsheets, compilers, and other products. These companies know that C produces compact and efficient programs. More important, they know that these programs will be easy to modify and easy to adapt to new models of computers.

Figure 1.2. Where C is used.

What's good for companies and C veterans is good for other users, too. More and more computer users have turned to C to secure its advantages for themselves. You don't have to be a computer professional to use C.

In the 1990s, many software houses began turning to the C++ language for large programming projects. C++ grafts object-oriented programming tools to the C language. (Object-oriented programming is a philosophy that attempts to mold the language to fit a problem instead of molding the problem to fit the language.) C++ is nearly a superset of C, meaning that any C program is, or nearly is, a valid C++ program, too. By learning C, you also learn much of C++.

Despite the popularity of newer languages, such as C++ and Java, C remains a core skill in the software business, typically ranking in the top 10 of desired skills. In particular, C has become popular for programming embedded systems. That is, it's used to program the increasingly common microprocessors found in automobiles, cameras, DVD players, and other modern conveniences. Also, C has been making inroads in FORTRAN's long dominance of scientific programming. Finally, as befits a language created to develop an operating system, it plays a strong role in the development of Linux. Thus, the first decade of the twenty-first century finds C still going strong.

In short, C is one of the most important programming languages and will continue to be so. If you want a job writing software, one of the first questions you should be able to answer yes to is "Oh say, can you C?"

Feed users, click here to go to this topic's page. You can find related topics and reference links and you can read the comments for this topic

Feed users, click here to post a comment for this topic

Note: There might be problems about HTML code and parsing and displaying HTML code of your viewer application. Also, there are problems caused by Google Reader on displaying of code samples on the entry. Also, the entry might be updated. So, it is recommended to visit the page of the entry.

Türkçe Karakterli Domain'lerin İç Yüzü

Bilal Hatipoglu — 22.12.2007 13:09:31

Haberiniz var mı? 4 Aralık 2006 tarihinden itibaren Türkçe karakterli domainler de tescil edilmeye başlandı. Almadıysanız ve kaptırmadıysanız şirketinizin veya isminizin domainini hemen alın! Ancak, bunu yaparken dikkat etmeniz gereken bir husus var! Türkiye'de, her şeyde olduğu gibi, bu yenilikten de rant elde etmeye çalışanlar, hatta şimdiye kadar bile büyük paralar götürenler var...

Evet, önce nasıl oldu da düne kadar sadece ascii karakterler içeren domainler alınabilirken bir anda Türkçe karakter desteği geldi ona bakalım:

Türkiye'de ".tr" uzantılı domainlerin kayıt edilmesinden sorumlu firma nic.tr sitesinde bu konuyla ilgili bilgilendirici bir yazı yazmış, önce oradan alıntı yapalım:

Internationalized Domain Names (Uluslararasılaştırılmış Alan Adları)

IDN nedir?

IDN, ASCII olmayan karakterleri içerebilen alan adlarıdır. Standart Internet alan adları çok kısıtlı bir karakter kümesini destekler: a-z, A-Z, 0-9 ve '-''den başka hiçbir karakter içeremez. Bu kısıtlama, kişinin kendi dilinde alan adları almasına büyük oranda engel teşkil eder.

IDN alt yapısı ve işleyişi

IDN standardı, var olan alan adı sistemini yenilemeye değil, genişletmeye yöneliktir. Alan adlarını birbirine dönüştürecek algoritmalar sunar. Standardın sunduğu algoritmalar Unicode karakterlerinden oluşan bir alan adını sadece ASCII karakterlerinden oluşan bir alan adına dönüştürebilir veya ASCII hali verilmiş bir IDN'i Unicode karakterlerden oluşan hale çevirebilir. Bir kullanıcı tarayıcısına 'çağlayan.web.tr' yazdığında, tarayıcı IDN standartlarında belirtilen algoritmaları kullanarak 'çağlayan.web.tr' alan adının ASCII karakterlerinden oluşan halini bulur ve DNS'e ASCII halini, sıradan bir alan adını sorgular gibi, sorgular. IDN, var olan yapıyı değiştirmemek için dönüştürme işini istemciye aktarır (bu örnekte tarayıcıya). IDN kaydı sırasında kullanıcı aslında -tam olarak- istediği alan adını almaz! Sistem kullanıcının adına IDN'in ASCII halini kaydeder. Kullanıcı 'çağlayan.web.tr' yi kaydettirdiğinde gerçekte 'xn--alayan-vua36b.web.tr' alan adını almış olur.

IDN' lerin düzgün çalışması için istemcilerin desteklemesi (istemcilerin bir IDN girildiğinde onun IDN olduğunu anlayıp ASCII haline çevirebilmesi) gerekir. IDN için şu anda popüler tarayıcıların çoğu destek sunmaktadır. Firefox ve IE7.0 tam destek sunarken, IE6.0 için bir yama yapılması gerekir. E-posta istemcileri henüz tam destek vermediklerinden, IDN'leri elektronik posta için kullanmanız tavsiye edilmemektedir. İstemciler IDN'i destekledikleri takdirde, e-posta ve diğer işlemler için de kullanılabilirler.

IDN ne değildir?

IDN var olan alan adı yapısını değiştirmeyi hedefleyen bir sistem değildir!

IDN'in amacı, varolan sistemde kaydedilemeyen ve Türkçe karekterleri içeren alan adlarını da kaydedebilmektir. Bu nedenle IDN, düzgün çalışan alt sisteme (DNS alt yapısı, internet standartları vb.) ek bir yük getirmez. Onun üstünde bir katman olarak iş görür.

Evet, buradan çıkarılması gereken önemli noktalar şunlar:

1- IDN var olan alan adı yapısını değiştirmeyi hedefleyen bir sistem değildir! Dünyadaki bütün DNS sistemi bizim Türkçe karakterlerimizi desteklemek için değişmedi :) Bu sadece bir algoritma sayesinde Türkçe karakterli isimlerin ascii karaktere çevrilmesidir.

2- Yapılan iş, sadece, web tarayıcınıza yazdığınız Türkçe karakterler içeren bir adresin, web tarayıcı tarafından bir algoritmayla ascii karakterlere sahip karşılığına çevrilmesidir.

3- Internet Explorer 6 henüz bunu desteklememektedir. Bunun için yama gerekmektedir. Mozilla ve IE 7 ise desteklemektedir.

4- E-Posta istemcileri (örn: Outlook) henüz bunu desteklememektedir. (Aslında kısa sürede onlar için de yama çıkar. Sonuçta basit bir şekilde isimleri dönüştürecek olan server tarafı değil, client tarafındaki uygulamadır)

Evet, şimdi bizim Türklerin bundan nasıl rant sağlamaya çalıştıklarına bakalım:

1- Şu an itibariyle bir çok şirket ve şahsın ismi, bu sistem daha devreye girmeden satın alınmıştır ve bir çoğu milyarlarca lira (binlerce YTL) ücretle satılmaya çalışılmaktadır.

2- Şu anda "Türkçe karakterli domain satıyoruz" diyen firmalar normalde 7-8 dolar olan domainleri 20 dolara satmaya çalışmaktadır.

Arkadaşlar, bu hususlara dikkat. Birileri bu işten ciddi miktarda kar elde edecek. Buna mahal vermeyelim. Peki ne yapacağız?

Türkçe karakterli domain almak istiyorsak ne yapmalıyız?

20 dolar verip Türkçe karakterli bir domain almamalıyız! Bunun yerine, Mozilla veya IE 7'yi çalıştırıp, almak istediğimiz domaini yazıp (örn: www.bhatipoğlu.com) ilgini web tarayıcının, algoritmayı kullanarak saliseler içinde bunu ascii karakterli haline çevirmesini bekleyip, çevirdikten sonra bu ascii karakterli halini satın almalıyız! (örn: bhatipoğlu.com için xn--bhatipolu-wkb.com)

Böylece, bu ismi 7-8 dolar(yıllık) gibi bir ücretle satın almış oluruz. Zaten "Türkçe karakterli domain satıyoruz" diyen firmaların yaptığı iş de bu!

Feed users, click here to go to this topic's page. You can find related topics and reference links and you can read the comments for this topic

Feed users, click here to post a comment for this topic

Note: There might be problems about HTML code and parsing and displaying HTML code of your viewer application. Also, there are problems caused by Google Reader on displaying of code samples on the entry. Also, the entry might be updated. So, it is recommended to visit the page of the entry.

Windows Source Codes

Bilal Hatipoglu — 17.09.2007 00:00:30

Microsoft has suffered what appears to be a severe leak of Windows source code, with a file circulating on the Internet appearing to consist of several million lines of code from around mid-2000. The source code seems to relate to NT4 and Windows 2000, and in a statement the company has conceded that "portions of the Microsoft Windows 2000 and Windows NT 4.0 source code were illegally made available on the Internet.

Finally, we got the source code with great effort, and I am publishing just some part of it.

"It's illegal for third parties to post Microsoft source code," spokesman Tom Pilla said. "We obviously take that very seriously." So, don't tell anybody that you got it from here, just Enjoy ;)

/* Source Code Windows 2000 */

#include "win31.h"
#include "win95.h"
#include "win98.h"
#include "workst~1.h"
#include "evenmore.h"
#include "oldstuff.h"
#include "billrulz.h"
#include "monopoly.h"
#include "backdoor.h"
#define INSTALL = HARD

char make_prog_look_big(16000000);
void main()
{
   while(!CRASHED)
   {
     display_copyright_message();
     display_bill_rules_message();
     do_nothing_loop();

     if (first_time_installation)
       {
       make_100_megabyte_swapfile();
       do_nothing_loop();
       totally_screw_up_HPFS_file_system();
       search_and_destroy_the_rest_of-OS2();
       make_futile_attempt_to_damage_Linux();
       disable_Netscape();
       disable_RealPlayer();
       disable_Lotus_Products();
       hang_system();
       } //if
     write_something(anything);
     display_copyright_message();
     do_nothing_loop();
     do_some_stuff();

     if (still_not_crashed)
     {
     display_copyright_message();
     do_nothing_loop();
     basically_run_windows_31();
     do_nothing_loop();
     } // if
   } //while

   if (detect_cache())
     disable_cache();

   if (fast_cpu())
     {
     set_wait_states(lots);
     set_mouse(speed,very_slow);
     set_mouse(action,jumpy);
     set_mouse(reaction,sometimes);
     } //if

   /* printf("Welcome to Windows 3.1");    */
   /* printf("Welcome to Windows 3.11");   */
   /* printf("Welcome to Windows 95");     */
   /* printf("Welcome to Windows NT 3.0"); */
   /* printf("Welcome to Windows 98");     */
   /* printf("Welcome to Windows NT 4.0"); */
   printf("Welcome to Windows 2000");

   if (system_ok())
     crash(to_dos_prompt)
   else
     system_memory = open("a:\swp0001.swp",O_CREATE);

   while(something)
     {
     sleep(5);
     get_user_input();
     sleep(5);
     act_on_user_input();
     sleep(5);
     } // while
   create_general_protection_fault();

} // main

Decode Function in Oracle SQL

Bilal Hatipoglu — 16.09.2007 23:24:21

The decode function can be used in SQL for and IF-THEN-ELSE construction. It's an alternative for the CASE statement which was introduced in Oracle 8.

Decode is a very useful and handy function for Oracle queries. It replaces the complex If-Then-Else logic, which is used to display different things based on different values in a column. This is usually used for pivoting purposes.

Syntax is like:

DECODE(expr, search, result
             [, search, result ]...
       [, default ]
      )

DECODE compares expr to each search value one by one. If expr is equal to a search, then Oracle Database returns the corresponding result. If no match is found, then Oracle returns default. If default is omitted, then Oracle returns null.

The arguments can be any of the numeric types (NUMBER, BINARY_FLOAT, or BINARY_DOUBLE) or character types.

The search, result, and default values can be derived from expressions. Oracle Database uses short-circuit evaluation. That is, the database evaluates each search value only before comparing it to expr, rather than evaluating all search values before comparing any of them with expr. Consequently, Oracle never evaluates a search if a previous search is equal to expr.

Oracle automatically converts expr and each search value to the datatype of the first search value before comparing. Oracle automatically converts the return value to the same datatype as the first result. If the first result has the datatype CHAR or if the first result is null, then Oracle converts the return value to the datatype VARCHAR2.

In a DECODE function, Oracle considers two nulls to be equivalent. If expr is null, then Oracle returns the result of the first search that is also null.

The maximum number of components in the DECODE function, including expr, searches, results, and default, is 255.

An Example:

You could use the decode function in an SQL statement as follows:

SELECT supplier_name, 
decode(supplier_id, 10000, 'IBM', 
 10001, 'Microsoft', 
 10002, 'Hewlett Packard', 
  'Gateway') result 
FROM suppliers; 

The above decode statement is equivalent to the following IF-THEN-ELSE statement:

IF supplier_id = 10000 THEN
     result := 'IBM';

ELSIF supplier_id = 10001 THEN
    result := 'Microsoft';

ELSIF supplier_id = 10002 THEN
    result := 'Hewlett Packard';

ELSE
    result := 'Gateway';

END IF;

The decode function will compare each supplier_id value, one by one.

More Examples:

Decode Demo #1
Decode Demo #2
Decode Demo #3
Decode Demo #4

DECODE vs. CASE

CASE can evaluate any expression, but DECODE is limited to compare discrete values. But CASE is more compex, has more code and less readable I think, so if you can do your job with simple DECODE, do it with DECODE.

Before version 8.1, the DECODE was the only thing providing IF-THEN-ELSE functionality in Oracle SQL. Because DECODE can only compare discrete values (not ranges), continuous data had to be contorted into discreet values using functions like FLOOR and SIGN. In version 8.1, Oracle introduced the searched CASE statement, which allowed the use of operators like > and BETWEEN (eliminating most of the contortions) and allowing different values to be compared in different branches of the statement (eliminating most nesting). In version 9.0, Oracle introduced the simple CASE statement, that reduces some of the verbosity of the CASE statement, but reduces its power to that of DECODE.

Another difference is, CASE is an ANSI standard, where as Decode is proprietary for Oracle.

Then comes 11g!

Traditional pivoting is done with DECODE and CASE in 10g or earlier. In 11g, there are two new operators: PIVOT and UNPIVOT. They can be used for pivoting. For more information refer to Reference links below.

Feed users, click here to go to this topic's page. You can find related topics and reference links and you can read the comments for this topic

Feed users, click here to post a comment for this topic

Note: There might be problems about HTML code and parsing and displaying HTML code of your viewer application. Also, there are problems caused by Google Reader on displaying of code samples on the entry. Also, the entry might be updated. So, it is recommended to visit the page of the entry.

Hello World!

Bilal Hatipoglu — 16.09.2007 22:32:48

A "hello world" program is a computer program that prints out "Hello, World!" on a display device. It is used in many introductory tutorials for teaching a programming language. Such a program is typically one of the simplest programs possible in a computer language. Some are surprisingly complex, especially in some graphical user interface (GUI) contexts, but most are very simple, especially those which rely heavily on a particular command line interpreter ("shell") to perform the actual output. In many embedded systems, the text may be sent to a one or two-line liquid crystal display (LCD), or some other appropriate signal, such as an LED being turned on, may substitute for the message.

A "hello world" program can be a useful sanity test to make sure that a language's compiler, development environment, and run-time environment are correctly installed. Configuring a complete programming toolchain from scratch to the point where even trivial programs can be compiled and run can involve substantial amounts of work. For this reason, a simple program is used first when testing a new tool chain.

Here is an "Hello World" program collection that includes 338 Hello World programs in many more-or-less well known programming languages:

http://www.roesler-ac.de/wolfram/hello.htm

I didn't know there exists that much programming languages :) I think this includes not only the programming languages, but whatever that can write any output to anywhere :)

And here is a funny joke from GNU: How the way people code "Hello World" varies depending on their age and job:

http://www.gnu.org/fun/jokes/helloworld.html

The chief executive version is really funny, isn't it? :)

Feed users, click here to go to this topic's page. You can find related topics and reference links and you can read the comments for this topic

Feed users, click here to post a comment for this topic

Note: There might be problems about HTML code and parsing and displaying HTML code of your viewer application. Also, there are problems caused by Google Reader on displaying of code samples on the entry. Also, the entry might be updated. So, it is recommended to visit the page of the entry.

Turkcell Staj Günlüğü - 12: Partitioning

Bilal Hatipoglu — 06.09.2007 12:35:28

Merhaba, yine uzun bir aradan sonra staj günlüğüme yazmaya devam ediyorum. Bu, serinin 12. yazısı olacak ve konumuz Ertürk'ün yaptığı partitioning sunumu.

Turkcell'deki stajımızın sonuna yaklaşıyoruz yavaş yavaş, son haftalarda da gayet yoğun projeler üzerinde uğraşıyorduk herbirimiz. Dolayısıyla yazmaya vaktim olmadı. Stajın benim açımdan (ve sanırım Yazlım Geliştirme departmanındaki diğer arkadaşlar açısından da) çok keyifli geçtiğini söyleyebilirim. Hepimiz uzun soluklu, gerçek projelerde yer aldık ve kod yazdık. Eğitici sunumlar yaz boyunca devam etti, ediyor. Ve en güzeli de buradaki üst düzey yöneticilerimizle sohbet toplantılarımız oluyor ve onlar da bizimle tecrübelerini paylaşıyorlar. Gerçekten çok keyifli herşey. Burada öğrendiklerimiz ve özellikle yöneticilerimizle sohbetlerimiz binlerce dolar vererek bile kazanılamayacak tecrübelerdi. Turkcell Türkiye'nin en büyük ve en gelişmiş platformarından birine sahip ve böyle bir platformda çalışma, proje geliştirme, analiz yapma imkanını başka bir şekilde veya her zaman bulamazsınız.

Bu imkanları bize sağlayan, staj dönemini keyiflendirerek hem Turkcell'e hem de stajyerlere çok büyük katkıda bulunan bu süreci destekleyen herkese (başta Tonguç abi olmaz üzere) teşekkür ediyorum. Neyse şimdilik bu kadar yeter, stajın sonuna doğru daha romantik (:P) şeyler yazmaya devam ederim nasılsa :) Biz Oracle'a dönelim, sonuçta Kategori olarak Oracle yazdık bu makaleyi :)

Ertürk'ün parititoning sunumu üzerinden çok zaman geçti, ama hala gözümün önünde sanki :) Ben eğitim odasında kapının önünde, Ertürk'ün sağında oturuyordum. Sunuma da tekrar göz atınca oldukça fazla yazacak şey çıktı yine.

Öncelikle Ertürk'ün sunumunu buradan indirebilirsiniz. Sunumu ve aşağıda "Links & References" bölümğnde verdiğim linkleri gözden geçirerek en azından başlangıç seviyesinde bir çok şey öğrenebilirsiniz.

• Partitioning'de de esas amaç her zamanki gibi performans :) Mantık "Divide & Conquer" (Böl ve fethet)'e dayanıyor. Tabloları partition'lara ayırarak, verileri bir ilişkiye göre gruplayarak, performanstan kazanmak mümkün. Ama elbette belirli koşullarda. Performanstan kazanmanın yanında bir amaç daha var, o da yönetimi kolaylaştırmak. Tabloların yönetimini partitionlarla kolaylaştırmak mümkün. Daha ilerde bahsedeceğimiz Oracle'ın partition'lar için sunduğu çok esnek imkanlar var.

• Partitioning daha çok Datawarehouse'larda kullanılabilecek bir özellik. Çünkü OLTP sistemlerde zaten az müktarda veri okunur ve genelde indexler yardımıyla veriye zaten hızlı şekilde ulaşırız. Fakat OLAP sistemlerde Full Table Scan(FTS)'ler havalarda uçuşur :)) Bütün tablolar okunarak işlenir, ki bunalr zaten büyük tablolardır. Bizim ihtiyacımız olan bilgiye partitioning yardımıyla çok daha az disk okumasıyla ulaşabiliriz.

• Partitioning tipleri var. Bunlardan bir tanesi Range Partitioning. Mantık çok basit, veriyi belli değer aralıklarına göre gruplamak, bunları farklı partitionlarda saklamak. Örneğin, tarihe göre, her senenin versini farklı bir partition'da tutmak gibi. Bu sayede sadece bir kısım veriye ihtiyacımız olduğunda bütün tabloyu okumaktan kurtulmuş, sadece ihtiyacımız olan partition'ı okumuş oluruz. Bu tür, iş mantığına uyması açısından güzel.

• Range partitioning'de bir MAXVALUE değeri vermemiz gerekiyor, çünkü her partition belli bir range'e düşmesi lazım.

• Bir diğer tür de Hash Partitioning. Bunda da kullanılan bir hash fonksiyonuna göre veri gruplanıyor. Bu şekilde de okumak istediğimiz verinin kriterini aynı hash'ten geçirerek hangi partition'dan okuaycağımızı anlayabiliyoruz(daha doğrusu Oracle bunu anlayıp hızlı bir şekilde veriyi önümüze getiriyor).

• List partitioning de ise hangi değerlerin hangi partition'a düşeceğini kendimiz liste alinde belirliyoruz. Bu biraz daha custom bir uygulama.

• Composite partitioning ise, Range ile ayrılan partition'lar içinde birlikte list veya hash partitioning'in de kullanılmasıdır. Bunlara subpartition denir.

• Partition edilmiş tablolarda local indexler ve/veya global indexler olabilir. Local indexler, her partition'ın kendi içindeki indexidir. Global index ise, bütün partitionları yani bütün tabloyu kapsayan indexlerdir. İkisinin de kullanım alanları farklıdır, örneğin partition'lı bir tablo üzerindeki primary key index'i global ve partition'sız olmaz zorunda.

• Tablolar gibi indexler de partitionlanabilir. Aynı mantıkla, index'i de belli değerlere göre gruplamış oluyoruz partition'layarak.

• Partitioning, çok faydalı, güzel bir özellik fakat doğru kullanılmadığından çok ciddi sorunlara ve performasn kayıplarına da neden olabiliyor. Partitioning de Oracle'ın sağlamış olduğu güzelliklerden bir tanesi. Bu konuda temel düzeyde bilgi sahibi olup bunu da alet çantamıza atmakta fayda var :) Çünkü çok özel ihtiyaçlar için çok özel çözümler olabiliyor. Bunları önceden kestirmek, bilmek ve uygulamak önemli.

Ertürk'ün sunumundan notlarım da bu kadardı. Hepinize iyi çalışmalar.

Feed users, click here to go to this topic's page. You can find related topics and reference links and you can read the comments for this topic

Feed users, click here to post a comment for this topic

Note: There might be problems about HTML code and parsing and displaying HTML code of your viewer application. Also, there are problems caused by Google Reader on displaying of code samples on the entry. Also, the entry might be updated. So, it is recommended to visit the page of the entry.

Examining show_space

Bilal Hatipoglu — 13.08.2007 20:20:38

Many people knows the show_space procedure of Tom KYTE. The procedure shows the space issues for an object, given in parameters. Execution of procedure is simple, and statistics are good. But, in AskTom, the show_space procedure has many versions :) and it's hard to know which is the last version. Also, the procedures can thow some errors in some cases and it is sometimes hard to clarify the error. In this topic, I will deal with them and give the code of last-version, running show_space code :)

First, let's create the test stuff:

SQL*Plus: Release 10.2.0.1.0 - Production on Mon Aug 13 12:22:17 2007

Copyright (c) 1982, 2005, Oracle.  All rights reserved.

SQL> conn sys@bilalEE as sysdba;

Enter password:

Connected to:
Oracle Database 10g Enterprise Edition Release 10.2.0.1.0 - Production
With the Partitioning, OLAP and Data Mining options

SQL> create user testspace identified by test;

User created.

SQL> grant resource, connect to testspace;

Grant succeeded.

SQL> conn testspace/test@bilalEE
Connected.
SQL> create table tspace as select * from all_objects;

Table created.

SQL> create table tspace_p partition by hash (owner) as select * from all_objects;

Table created.

SQL>

Download Code

So far, we have created a test user called testsapce and two tables in its schema, first, a normal table tspace and a partitioned table tspace_t. Let's start with the oldest version of show_space that I found on AskTom:

SQL*Plus: Release 10.2.0.1.0 - Production on Mon Aug 13 12:28:20 2007

Copyright (c) 1982, 2005, Oracle.  All rights reserved.

SQL> conn sys@bilalEE as sysdba;

Enter password:

Connected to:
Oracle Database 10g Enterprise Edition Release 10.2.0.1.0 - Production
With the Partitioning, OLAP and Data Mining options

SQL> create or replace
  2    procedure show_space
  3    ( p_segname in varchar2,
  4      p_owner   in varchar2 default user,
  5      p_type    in varchar2 default 'TABLE',
  6      p_free   in BOOLEAN default true )
  7    as
  8        l_free_blks                 number;
  9
 10        l_total_blocks              number;
 11       l_total_bytes               number;
 12       l_unused_blocks             number;
 13       l_unused_bytes              number;
 14        l_LastUsedExtFileId         number;
 15        l_LastUsedExtBlockId        number;
 16        l_LAST_USED_BLOCK           number;
 17        procedure p( p_label in varchar2, p_num in number )
 18        is
 19        begin
 20            dbms_output.put_line( rpad(p_label,40,'.') ||
 21                                  p_num );
 22        end;
 23    begin
 24       if (p_free) then
 25        dbms_space.free_blocks
 26        ( segment_owner     => p_owner,
 27          segment_name      => p_segname,
 28          segment_type      => p_type,
 29          freelist_group_id => 0,
 30          free_blks         => l_free_blks );
 31       end if;
 32        dbms_space.unused_space
 33        ( segment_owner     => p_owner,
 34          segment_name      => p_segname,
 35          segment_type      => p_type,
 36          total_blocks      => l_total_blocks,
 37          total_bytes       => l_total_bytes,
 38          unused_blocks     => l_unused_blocks,
 39          unused_bytes      => l_unused_bytes,
 40          LAST_USED_EXTENT_FILE_ID => l_LastUsedExtFileId,
 41          LAST_USED_EXTENT_BLOCK_ID => l_LastUsedExtBlockId,
 42          LAST_USED_BLOCK => l_LAST_USED_BLOCK );
 43
 44        p( 'Free Blocks', l_free_blks );
 45        p( 'Total Blocks', l_total_blocks );
 46        p( 'Total Bytes', l_total_bytes );
 47        p( 'Unused Blocks', l_unused_blocks );
 48        p( 'Unused Bytes', l_unused_bytes );
 49        p( 'Last Used Ext FileId', l_LastUsedExtFileId );
 50        p( 'Last Used Ext BlockId', l_LastUsedExtBlockId );
 51        p( 'Last Used Block', l_LAST_USED_BLOCK );
 52    end;
 53    /

Procedure created.

SQL>
SQL> grant execute on show_space to public;

Grant succeeded.

SQL>
SQL> create or replace public synonym show_space for sys.show_space;

Synonym created.

SQL>

Download Code

I created the procedure on sys schema and grant execute to public plus a public synonym.

Can you see the problems in this code?

First, we have a security issue. By executing this procedure, all users can be able to see the statistical information about the tables on sys schema *without having any grant*. Let's see:

SQL*Plus: Release 10.2.0.1.0 - Production on Mon Aug 13 12:34:53 2007

Copyright (c) 1982, 2005, Oracle.  All rights reserved.

SQL> conn sys@bilalEE as sysdba;

Enter password:

Connected to:
Oracle Database 10g Enterprise Edition Release 10.2.0.1.0 - Production
With the Partitioning, OLAP and Data Mining options

SQL> create table t1 as select object_name from all_objects;

Table created.

SQL> conn testspace/test@bilalEE
Connected.
SQL> exec show_space('T1','SYS');

PL/SQL procedure successfully completed.

SQL> set serveroutput on
SQL> exec show_space('T1','SYS');
Free Blocks.............................0
Total Blocks............................256
Total Bytes.............................2097152
Unused Blocks...........................53
Unused Bytes............................434176
Last Used Ext FileId....................1
Last Used Ext BlockId...................60553
Last Used Block.........................75

PL/SQL procedure successfully completed.

SQL>

How?? The procedure is actually in sys schema, and unless you specify, it will run with the rights of the procedure's owner. (Look at the AUTHID section at: http://www.psoug.org/reference/procedures.html)

So, we have to specify AUTHID CURRENT_USER pragma.

Also, another problem with the above procedure is, it does not handle the segments in tablespaces with AUTO SEGMENT SPACE MANAGEMENT, that is default, so we will usually get this error:

SQL*Plus: Release 10.2.0.1.0 - Production on Mon Aug 13 13:21:24 2007

Copyright (c) 1982, 2005, Oracle.  All rights reserved.

SQL> conn testspace/test@bilalEE
Connected.
SQL> exec show_space('TSPACE');
BEGIN show_space('TSPACE'); END;

*
ERROR at line 1:
ORA-10618: Operation not allowed on this segment
ORA-06512: at "SYS.DBMS_SPACE", line 152
ORA-06512: at "SYS.SHOW_SPACE", line 24
ORA-06512: at line 1

SQL>

Also, this procedure does not support partitioned tables. The second version of the show_space procedure is like this:

SQL*Plus: Release 10.2.0.1.0 - Production on Mon Aug 13 13:24:59 2007

Copyright (c) 1982, 2005, Oracle.  All rights reserved.

SQL> conn sys@bilalEE as sysdba
Enter password:
Connected.
SQL> create or replace
  2  procedure show_space
  3  ( p_segname in varchar2,
  4    p_owner   in varchar2 default user,
  5    p_type    in varchar2 default 'TABLE',
  6    p_partition in varchar2 default NULL )
  7  authid current_user
  8  as
  9      l_free_blks                 number;
 10
 11      l_total_blocks              number;
 12      l_total_bytes               number;
 13      l_unused_blocks             number;
 14      l_unused_bytes              number;
 15      l_LastUsedExtFileId         number;
 16      l_LastUsedExtBlockId        number;
 17      l_LAST_USED_BLOCK           number;
 18      procedure p( p_label in varchar2, p_num in number )
 19      is
 20      begin
 21          dbms_output.put_line( rpad(p_label,40,'.') ||
 22                                p_num );
 23      end;
 24  begin
 25      for x in ( select tablespace_name
 26                   from dba_tablespaces
 27                  where tablespace_name = ( select tablespace_name
 28                                              from dba_segments
 29                                             where segment_type = p_type
 30                                               and segment_name = p_segname
 31                                    and SEGMENT_SPACE_MANAGEMENT <> 'AUTO'
 32                                    and rownum=1 )
 33               )
 34      loop
 35      dbms_space.free_blocks
 36      ( segment_owner     => p_owner,
 37        segment_name      => p_segname,
 38        segment_type      => p_type,
 39        partition_name    => p_partition,
 40        freelist_group_id => 0,
 41        free_blks         => l_free_blks );
 42      end loop;
 43
 44      dbms_space.unused_space
 45      ( segment_owner     => p_owner,
 46        segment_name      => p_segname,
 47        segment_type      => p_type,
 48            partition_name    => p_partition,
 49        total_blocks      => l_total_blocks,
 50        total_bytes       => l_total_bytes,
 51        unused_blocks     => l_unused_blocks,
 52        unused_bytes      => l_unused_bytes,
 53        LAST_USED_EXTENT_FILE_ID => l_LastUsedExtFileId,
 54        LAST_USED_EXTENT_BLOCK_ID => l_LastUsedExtBlockId,
 55        LAST_USED_BLOCK => l_LAST_USED_BLOCK );
 56
 57      p( 'Free Blocks', l_free_blks );
 58      p( 'Total Blocks', l_total_blocks );
 59      p( 'Total Bytes', l_total_bytes );
 60      p( 'Total MBytes', trunc(l_total_bytes/1024/1024) );
 61      p( 'Unused Blocks', l_unused_blocks );
 62      p( 'Unused Bytes', l_unused_bytes );
 63      p( 'Last Used Ext FileId', l_LastUsedExtFileId );
 64      p( 'Last Used Ext BlockId', l_LastUsedExtBlockId );
 65      p( 'Last Used Block', l_LAST_USED_BLOCK );
 66  end;
 67  /

Procedure created.

SQL>

Download Code

This version solves the partition problem and security problem. But we already don't have support for AUTO SEGMENT SPACE MANAGEMENT:

SQL*Plus: Release 10.2.0.1.0 - Production on Mon Aug 13 13:29:21 2007

Copyright (c) 1982, 2005, Oracle.  All rights reserved.

SQL> conn testspace/test@bilalEE
Connected.
SQL> exec show_space('T1','SYS');
BEGIN show_space('T1','SYS'); END;

*
ERROR at line 1:
ORA-00942: table or view does not exist
ORA-06512: at "SYS.SHOW_SPACE", line 24
ORA-06512: at line 1

SQL>

And the last, the most reliable version of show_space is like this one:

SQL*Plus: Release 10.2.0.1.0 - Production on Mon Aug 13 13:31:54 2007

Copyright (c) 1982, 2005, Oracle.  All rights reserved.

SQL> conn sys@bilalEE as sysdba
Enter password:
Connected.
SQL> create or replace procedure show_space
  2  ( p_segname in varchar2,
  3    p_owner   in varchar2 default user,
  4    p_type    in varchar2 default 'TABLE',
  5    p_partition in varchar2 default NULL )
  6  -- this procedure uses authid current user so it can query DBA_*
  7  -- views using privileges from a ROLE and so it can be installed
  8  -- once per database, instead of once per user that wanted to use it
  9  authid current_user
 10  as
 11      l_free_blks                 number;
 12      l_total_blocks              number;
 13      l_total_bytes               number;
 14      l_unused_blocks             number;
 15      l_unused_bytes              number;
 16      l_LastUsedExtFileId         number;
 17      l_LastUsedExtBlockId        number;
 18      l_LAST_USED_BLOCK           number;
 19      l_segment_space_mgmt        varchar2(255);
 20      l_unformatted_blocks number;
 21      l_unformatted_bytes number;
 22      l_fs1_blocks number; l_fs1_bytes number;
 23      l_fs2_blocks number; l_fs2_bytes number;
 24      l_fs3_blocks number; l_fs3_bytes number;
 25      l_fs4_blocks number; l_fs4_bytes number;
 26      l_full_blocks number; l_full_bytes number;
 27
 28      -- inline procedure to print out numbers nicely formatted
 29      -- with a simple label
 30      procedure p( p_label in varchar2, p_num in number )
 31      is
 32      begin
 33          dbms_output.put_line( rpad(p_label,40,'.') ||
 34                                to_char(p_num,'999,999,999,999') );
 35      end;
 36  begin
 37     -- this query is executed dynamically in order to allow this procedure
 38     -- to be created by a user who has access to DBA_SEGMENTS/TABLESPACES
 39     -- via a role as is customary.
 40     -- NOTE: at runtime, the invoker MUST have access to these two
 41     -- views!
 42     -- this query determines if the object is a ASSM object or not
 43     begin
 44        execute immediate
 45            'select ts.segment_space_management
 46               from dba_segments seg, dba_tablespaces ts
 47              where seg.segment_name      = :p_segname
 48                and (:p_partition is null or
 49                    seg.partition_name = :p_partition)
 50                and seg.owner = :p_owner
 51                and seg.tablespace_name = ts.tablespace_name'
 52               into l_segment_space_mgmt
 53              using p_segname, p_partition, p_partition, p_owner;
 54     exception
 55         when too_many_rows then
 56            dbms_output.put_line
 57            ( 'This must be a partitioned table, use p_partition => ');
 58            return;
 59     end;
 60
 61
 62     -- if the object is in an ASSM tablespace, we must use this API
 63     -- call to get space information, else we use the FREE_BLOCKS
 64     -- API for the user managed segments
 65     if l_segment_space_mgmt = 'AUTO'
 66     then
 67       dbms_space.space_usage
 68       ( p_owner, p_segname, p_type, l_unformatted_blocks,
 69         l_unformatted_bytes, l_fs1_blocks, l_fs1_bytes,
 70         l_fs2_blocks, l_fs2_bytes, l_fs3_blocks, l_fs3_bytes,
 71         l_fs4_blocks, l_fs4_bytes, l_full_blocks, l_full_bytes, p_partition);
 72
 73       p( 'Unformatted Blocks ', l_unformatted_blocks );
 74       p( 'FS1 Blocks (0-25)  ', l_fs1_blocks );
 75       p( 'FS2 Blocks (25-50) ', l_fs2_blocks );
 76       p( 'FS3 Blocks (50-75) ', l_fs3_blocks );
 77       p( 'FS4 Blocks (75-100)', l_fs4_blocks );
 78       p( 'Full Blocks        ', l_full_blocks );
 79    else
 80       dbms_space.free_blocks(
 81         segment_owner     => p_owner,
 82         segment_name      => p_segname,
 83         segment_type      => p_type,
 84         freelist_group_id => 0,
 85         free_blks         => l_free_blks);
 86
 87       p( 'Free Blocks', l_free_blks );
 88    end if;
 89
 90    -- and then the unused space API call to get the rest of the
 91    -- information
 92    dbms_space.unused_space
 93    ( segment_owner     => p_owner,
 94      segment_name      => p_segname,
 95      segment_type      => p_type,
 96      partition_name    => p_partition,
 97      total_blocks      => l_total_blocks,
 98      total_bytes       => l_total_bytes,
 99      unused_blocks     => l_unused_blocks,
100      unused_bytes      => l_unused_bytes,
101      LAST_USED_EXTENT_FILE_ID => l_LastUsedExtFileId,
102      LAST_USED_EXTENT_BLOCK_ID => l_LastUsedExtBlockId,
103      LAST_USED_BLOCK => l_LAST_USED_BLOCK );
104
105      p( 'Total Blocks', l_total_blocks );
106      p( 'Total Bytes', l_total_bytes );
107      p( 'Total MBytes', trunc(l_total_bytes/1024/1024) );
108      p( 'Unused Blocks', l_unused_blocks );
109      p( 'Unused Bytes', l_unused_bytes );
110      p( 'Last Used Ext FileId', l_LastUsedExtFileId );
111      p( 'Last Used Ext BlockId', l_LastUsedExtBlockId );
112      p( 'Last Used Block', l_LAST_USED_BLOCK );
113  end;
114  /

Procedure created.


SQL> conn testspace/test@bilalEE
Connected.
SQL> exec show_space('TSPACE');
BEGIN show_space('TSPACE'); END;

*
ERROR at line 1:
ORA-00942: table or view does not exist
ORA-06512: at "SYS.SHOW_SPACE", line 44
ORA-06512: at line 1


SQL>

Download Code

Ups! We got an error? Actually, the error was about this query, in the code: "Line 45: 'select ts.segment_space_management from dba_segments seg, dba_tablespaces ts ..." We must give the grant to this dictionary tables to public, if we want to run this procedure. This is for determining whether the segment space management is auto or manual. Also it is another security issue, but not so critical if this is not producton system. (Don't give any user the right to create or execute the show_space procedure on your production system)

SQL*Plus: Release 10.2.0.1.0 - Production on Mon Aug 13 13:44:56 2007

Copyright (c) 1982, 2005, Oracle.  All rights reserved.

SQL> conn sys@bilalEE as sysdba
Enter password:
Connected.
SQL> grant select on dba_segments to testspace;

Grant succeeded.

SQL> grant select on dba_tablespaces to testspace;

Grant succeeded.

SQL> conn testspace/test@bilalEE
Connected.
SQL> set serveroutput on
SQL> exec show_space('TSPACE');
Unformatted Blocks .....................               0
FS1 Blocks (0-25)  .....................               0
FS2 Blocks (25-50) .....................               0
FS3 Blocks (50-75) .....................               0
FS4 Blocks (75-100).....................               0
Full Blocks        .....................             568
Total Blocks............................             640
Total Bytes.............................       5,242,880
Total MBytes............................               5
Unused Blocks...........................              54
Unused Bytes............................         442,368
Last Used Ext FileId....................               4
Last Used Ext BlockId...................          21,257
Last Used Block.........................              74

PL/SQL procedure successfully completed.

SQL>

At last, we can see a show_space output :)

Free Blocks...... Number of blocks on the freelist
Total Blocks..... Total blocks allocated to the table
Total Bytes...... Total bytes allocated to the table
Unused Blocks.... Blocks that have never contained data
Unused Bytes..... The above in bytes

This output is for a automatically managed segment(automatic segment space management). For a manually managed segment, we will have an output like this:

SQL*Plus: Release 10.2.0.1.0 - Production on Mon Aug 13 14:29:32 2007

Copyright (c) 1982, 2005, Oracle.  All rights reserved.

SQL> conn sys@bilalEE as sysdba
Enter password:
Connected.
SQL> set serveroutput on
SQL> exec show_space('T1');
Free Blocks.............................               0
Total Blocks............................             256
Total Bytes.............................       2,097,152
Total MBytes............................               2
Unused Blocks...........................              53
Unused Bytes............................         434,176
Last Used Ext FileId....................               1
Last Used Ext BlockId...................          60,553
Last Used Block.........................              75

PL/SQL procedure successfully completed.

SQL>

And for a partitioned table, we can get space statistics like this:

SQL*Plus: Release 10.2.0.1.0 - Production on Mon Aug 13 14:33:47 2007

Copyright (c) 1982, 2005, Oracle.  All rights reserved.

SQL> conn testspace/test@bilalEE
Connected.
SQL> exec show_space('TSPACE_P');
BEGIN show_space('TSPACE_P'); END;

*
ERROR at line 1:
ORA-14107: partition specification is required for a partitioned object
ORA-06512: at "SYS.DBMS_SPACE", line 175
ORA-06512: at "SYS.SHOW_SPACE", line 67
ORA-06512: at line 1


SQL> select partition_position, partition_name
  2  from   user_tab_partitions
  3  where  table_name = 'TSPACE_P';

PARTITION_POSITION PARTITION_NAME
------------------ ------------------------------
                 1 SYS_P21

SQL> set serveroutput on
SQL> exec show_space('TSPACE_P','TESTSPACE','TABLE PARTITION','SYS_P21');
Unformatted Blocks .....................               0
FS1 Blocks (0-25)  .....................               0
FS2 Blocks (25-50) .....................               0
FS3 Blocks (50-75) .....................               0
FS4 Blocks (75-100).....................               0
Full Blocks        .....................             568
Total Blocks............................             640
Total Bytes.............................       5,242,880
Total MBytes............................               5
Unused Blocks...........................              54
Unused Bytes............................         442,368
Last Used Ext FileId....................               4
Last Used Ext BlockId...................          21,897
Last Used Block.........................              74

PL/SQL procedure successfully completed.

SQL>

Also, you can use this one for partitioned tables. This package may be more useful:

SQL*Plus: Release 10.2.0.1.0 - Production on Mon Aug 13 14:56:50 2007

Copyright (c) 1982, 2005, Oracle.  All rights reserved.

SQL> conn sys@bilalEE as sysdba
Enter password:
Connected.
SQL> create or replace type show_space_type
  2  as object
  3  ( owner                 varchar2(30),
  4    segment_name          varchar2(30),
  5    partition_name        varchar2(30),
  6    segment_type          varchar2(30),
  7    free_blocks           number,
  8    total_blocks          number,
  9    unused_blocks         number,
 10    last_used_ext_fileid  number,
 11    last_used_ext_blockid number,
 12    last_used_block       number
 13  )
 14  /

Type created.

SQL> create or replace type show_space_table_type
  2  as table of show_space_type
  3  /

Type created.

SQL>
SQL>
SQL> create or replace
  2    function show_space_for
  3    ( p_segname   in varchar2,
  4      p_owner     in varchar2 default user,
  5      p_type      in varchar2 default 'TABLE',
  6      p_partition in varchar2 default NULL )
  7    return show_space_table_type
  8    authid CURRENT_USER
  9    PIPELINED
 10    as
 11        pragma autonomous_transaction;
 12        type rc is ref cursor;
 13        l_cursor rc;
 14
 15        l_free_blks                 number;
 16        l_total_blocks              number;
 17        l_total_bytes               number;
 18        l_unused_blocks             number;
 19        l_unused_bytes              number;
 20        l_LastUsedExtFileId         number;
 21        l_LastUsedExtBlockId        number;
 22        l_last_used_block           number;
 23        l_sql                       long;
 24        l_conj                       varchar2(7) default ' where ';
 25        l_owner varchar2(30);
 26        l_segment_name varchar2(30);
 27        l_segment_type varchar2(30);
 28        l_partition_name varchar2(30);
 29
 30        procedure add_predicate( p_name in varchar2, p_value in varchar2 )
 31        as
 32        begin
 33            if ( instr( p_value, '%' ) > 0 )
 34            then
 35                l_sql := l_sql || l_conj || p_name || ' like ''' ||
						upper(p_value) || '''';
 36                l_conj := ' and ';
 37            elsif ( p_value is not null )
 38            then
 39                l_sql := l_sql || l_conj || p_name || ' = ''' ||
						upper(p_value) || '''';
 40                l_conj := ' and ';
 41            end if;
 42        end;
 43    begin
 44        l_sql := 'select owner, segment_name, segment_type, partition_name
 45                    from dba_segments ';
 46
 47        add_predicate( 'segment_name', p_segname );
 48        add_predicate( 'owner', p_owner );
 49        add_predicate( 'segment_type', p_type );
 50        add_predicate( 'partition', p_partition );
 51
 52        execute immediate 'alter session set cursor_sharing=force';
 53        open l_cursor for l_sql;
 54        execute immediate 'alter session set cursor_sharing=exact';
 55
 56        loop
 57            fetch l_cursor into l_owner, l_segment_name, l_segment_type,
				 l_partition_name;
 58              dbms_output.put_line( l_segment_name || ',' || l_segment_type );
 59            exit when l_cursor%notfound;
 60            begin
 61            dbms_space.free_blocks
 62            ( segment_owner     => l_owner,
 63                segment_name      => l_segment_name,
 64                segment_type      => l_segment_type,
 65                partition_name    => l_partition_name,
 66                freelist_group_id => 0,
 67                free_blks         => l_free_blks );
 68
 69            dbms_space.unused_space
 70            ( segment_owner     => l_owner,
 71                segment_name      => l_segment_name,
 72                segment_type      => l_segment_type,
 73                partition_name    => l_partition_name,
 74                total_blocks      => l_total_blocks,
 75                total_bytes       => l_total_bytes,
 76                unused_blocks     => l_unused_blocks,
 77                unused_bytes      => l_unused_bytes,
 78                LAST_USED_EXTENT_FILE_ID => l_LastUsedExtFileId,
 79                LAST_USED_EXTENT_BLOCK_ID => l_LastUsedExtBlockId,
 80                LAST_USED_BLOCK => l_LAST_USED_BLOCK );
 81
 82        pipe row(show_space_type( l_owner, l_segment_name, l_partition_name,
 83              l_segment_type, l_free_blks, l_total_blocks,
                 l_unused_blocks, l_lastUsedExtFileId, 
 84              l_LastUsedExtBlockId, l_last_used_block ) );
 85            exception
 86                when others then null;
 87            end;
 88        end loop;
 89        close l_cursor;
 90
 91        return;
 92    end;
 93    /

Function created.

Download Code

Object-Oriented Features of Oracle - Part 3: Object Tables, Object Views and REFs

Bilal Hatipoglu — 13.08.2007 08:37:00

OBJECT TABLES

An object table is a special kind of table that holds objects and provides a relational view of the attributes of those objects.

Oracle lets you view this table in two ways:

• A single column table in which each entry is an object
• A multicolumn table in which each of the attributes of the object type occupies a column

You cannot see Object Tables in xxx_tables, use xxx_object_tables instead.

OBJECT VIEWS

An object view is a way to access relational data using object-relational features. It lets you develop object-oriented applications without changing the underlying relational schema.

Just as a view is a virtual table, an object view is a virtual object table.

Oracle provides object views as an extension of the basic relational view mechanism. By using object views, you can create virtual object tables from data—of either built-in or user-defined types—stored in the columns of relational or object tables in the database.

Object views provide the ability to offer specialized or restricted access to the data and objects in a database. For example, you can use an object view to provide a version of an employee object table that does not have attributes containing sensitive data and does not have a deletion method.

Object views allow the use of relational data in object-oriented applications. They let users:

• Try object-oriented programming techniques without converting existing tables
• Convert data gradually and transparently from relational tables to object-relational tables
• Use legacy RDBMS data with existing object-oriented applications

Advantages of Object Views

Using object views can lead to better performance. Relational data that make up a row of an object view traverse the network as a unit, potentially saving many round trips.

You can fetch relational data into the client-side object cache and map it into C or C++ structures so 3GL applications can manipulate it just like native structures.

Object views provide a gradual upgrade path for legacy data. They provide for co-existence of relational and object-oriented applications, and they make it easier to introduce object-oriented applications to existing relational data without having to make a drastic change from one paradigm to another.

Object views provide the flexibility of looking at the same relational or object data in more than one way. Thus you can use different in-memory object representations for different applications without changing the way you store the data in the database.

You can update, insert, and delete the data in an object view using the same SQL DML you use for object tables. Oracle updates the base tables of the object view if there is no ambiguity.

A view is not updatable if its view query contains joins, set operators, aggregate functions, GROUP BY, or DISTINCT. If a view query contains pseudocolumns or expressions, the corresponding view columns are not updatable. Object views often involve joins.

Why Use Object Views?

Just as a view is a virtual table, an object view is a virtual object table. Each row in the view is an object: you can call its methods, access its attributes using the dot notation, and create a REF that points to it.

Object views are useful in prototyping or transitioning to object-oriented applications because the data in the view can be taken from relational tables and accessed as if the table were defined as an object table. You can run object-oriented applications without converting existing tables to a different physical structure.

Object views can be used like relational views to present only the data that you want users to see. For example, you might create an object view that presents selected data from an employee table but omits sensitive data about salaries.

Using object views can lead to better performance. Relational data that make up a row of an object view traverse the network as a unit, potentially saving many round trips.

You can fetch relational data into the client-side object cache and map it into C structures or C++ or Java classes, so 3GL applications can manipulate it just like native classes. You can also use object-oriented features like complex object retrieval with relational data.

• By synthesizing objects from relational data, you can query the data in new ways. You can view data from multiple tables by using object de-referencing instead of writing complex joins with multiple tables.

• Because the objects in the view are processed within the server, not on the client, this can result in significantly fewer SQL statements and much less network traffic.

• The object data from object views can be pinned and used in the client side object cache. When you retrieve these synthesized objects in the object cache by means of specialized object-retrieval mechanisms, you reduce network traffic.

• You gain great flexibility when you create an object model within a view in that you can continue to develop the model. If you need to alter an object type, you can simply replace the invalidated views with a new definition.

• Using objects in views does not place any restrictions on the characteristics of the underlying storage mechanisms. By the same token, you are not limited by the restrictions of current technology. For example, you can synthesize objects from relational tables which are parallelized and partitioned.

• You can create different complex data models from the same underlying data.

REF's

A REF is a logical pointer to a row object that is constructed from the object identifier (OID) of the referenced object and is an Oracle built-in datatype. REFs and collections of REFs model associations among objects, particularly many-to-one relationships, thus reducing the need for foreign keys. REFs provide an easy mechanism for navigating between objects. You can use the dot notation to follow the pointers. Oracle does joins for you when needed, and in some cases can avoid doing joins.

You can use a REF to examine or update the object it refers to. You can also use a REF to obtain the object it refers to. You can change a REF so that it points to a different object of the same object type hierarchy or assign it a null value.

Scoped REFs

In declaring a column type, collection element, or object type attribute to be a REF, you can constrain it to contain only references to a specified object table. Such a REF is called a scoped REF. Scoped REF types require less storage space and allow more efficient access than unscoped REF types.

Dangling REFs

It is possible for the object identified by a REF to become unavailable through either deletion of the object or a revoking of privileges. Such a REF is called dangling. Oracle SQL provides a predicate (called IS DANGLING) to allow testing REFs for this condition.

Dangling REFs can be avoided by defining referential integrity constraints.

Dereference REFs

Accessing the object referred to by a REF is called dereferencing the REF. Oracle provides the DEREF operator to do this. Dereferencing a dangling REF results in a null object. Oracle provides implicit dereferencing of REFs.

<< Part1: Native Datatypes vs. Object Datatypes

<< Part2: Object Types and Collection Types

Object-Oriented Features of Oracle - Part 2: Object Types and Collection types

Bilal Hatipoglu — 13.08.2007 08:29:59

OBJECT TYPES

An object type differs from native SQL datatypes in that it is user-defined, and it specifies both the underlying persistent data (attributes) and the related behaviors (methods). Object types are abstractions of the real-world entities, for example, purchase orders. Object types store structured business data in its natural form and allow applications to retrieve it that way.

Object types and related object-oriented features, such as variable-length arrays and nested tables, provide higher-level ways to organize and access data in the database. Underneath the object layer, data is still stored in columns and tables, but you can work with the data in terms of the real-world entities--customers and purchase orders, for example--that make the data meaningful. Instead of thinking in terms of columns and tables when you query the database, you can simply select a customer.

Object types are abstractions of the real-world entities—for example, purchase orders—that application programs deal with. An object type is a schema object with three kinds of components:

• A name, which serves to identify the object type uniquely within that schema
• Attributes, which model the structure and state of the real-world entity. Attributes are built-in types or other user-defined types.
• Methods, which are functions or procedures written in PL/SQL or Java and stored in the database, or written in a language such as C and stored externally. Methods implement operations the application can perform on the real-world entity.

An object type is a template. A structured data unit that matches the template is called an object. An example object types and object instances:

Object Attributes and Methods

Attributes hold the data about an object's features of interest. For example, a student object type might have name, major, and graduation date attributes. An attribute has a declared datatype which can in turn be another object type. Taken together, the attributes of an object instance contain that object's data.

Methods are procedures or functions provided to enable applications to perform useful operations on the attributes of the object type. Methods are an optional element of an object type. They define the behavior of objects of that type and determine what (if anything) that type of object can do.

Types of Methods

Methods of an object type model the behavior of objects. The methods of an object type broadly fall into these categories:

• A Member method is a function or a procedure that always has an implicit SELF parameter as its first parameter, whose type is the containing object type.
• A Static method is a function or a procedure that does not have an implicit SELF parameter. Such methods can be invoked by qualifying the method with the type name, as in TYPE_NAME.METHOD. Static methods are useful for specifying user-defined constructors or cast methods.
• Comparison methods are used for comparing instances of objects.

Oracle supports the choice of implementing type methods in PL/SQL, Java, and C. Every object type also has one implicitly defined method that is not tied to specific objects, the object type's constructor method.

Object Type Constructor Methods

Every object type has a system-defined constructor method; that is, a method that makes a new object according to the object type's specification. The name of the constructor method is the name of the object type. Its parameters have the names and types of the object type's attributes. The constructor method is a function. It returns the new object as its value. You can also define your own constructor functions to use in place of the constructor functions that the system implicitly defines for every object type.

Comparison Methods

Methods play a role in comparing objects. Oracle has facilities for comparing two data items of a given built-in type (for example, two numbers), and determining whether one is greater than, equal to, or less than the other. Oracle cannot, however, compare two items of an arbitrary user-defined type without further guidance from the definer. Oracle provides two ways to define an order relationship among objects of a given object type: map methods and order methods.

Map methods use Oracle's ability to compare built-in types. Suppose that you have defined an object type called rectangle, with attributes height and width. You can define a map method area that returns a number, namely the product of the rectangle's height and width attributes. Oracle can then compare two rectangles by comparing their areas.

Order methods are more general. An order method uses its own internal logic to compare two objects of a given object type. It returns a value that encodes the order relationship. For example, it could return -1 if the first is smaller, 0 if they are equal, and 1 if the first is larger.

COLLECTION TYPES

Each collection type describes a data unit made up of an indefinite number of elements, all of the same datatype.

For modeling multi-valued attributes and many to many relationships, Oracle supports two collection datatypes: varrays and nested tables. Collection types can be used anywhere other datatypes can be used. You can have object attributes of a collection type in addition to columns of a collection type. For example, you might give a purchase order object type a nested table attribute to hold the collection of line items for a given purchase order. You use the CREATE TYPE statement to define collection types.

VARRAYs

An array is an ordered set of data elements. All elements of a given array are of the same datatype. Each element has an index, which is a number corresponding to the element's position in the array.

The number of elements in an array is the size of the array. Oracle allows arrays to be of variable size, which is why they are called VARRAYs. You must specify a maximum size when you declare the array type.

Creating an array type does not allocate space. It defines a datatype, which you can use as:

• The datatype of a column of a relational table
• An object type attribute
• A PL/SQL variable, parameter, or function return type.

A VARRAY is normally stored in line; that is, in the same tablespace as the other data in its row. If it is sufficiently large, however, Oracle stores it as a BLOB.

Nested Tables

A nested table is an unordered set of data elements, all of the same datatype. It has a single column, and the type of that column is a built-in type or an object type. If an object type, the table can also be viewed as a multicolumn table, with a column for each attribute of the object type. If compatibility is set to Oracle9i or higher, nested tables can contain other nested tables.

A table type definition does not allocate space. It defines a type, which you can use as:

• The datatype of a column of a relational table
• An object type attribute
• A PL/SQL variable, parameter, or function return type

When a table type appears as the type of a column in a relational table or as an attribute of the underlying object type of an object table, Oracle stores all of the nested table data in a single table, which it associates with the enclosing relational or object table.

A convenient way to access the elements of a nested table individually is to use a nested cursor.

Increasing the Size and Precision of VARRAYs and Nested Tables

When the element type of a VARRAY type is a variable character or RAW type or a numeric type, you can increase the size of the variable character or RAW type or increase the precision of the numeric type. A new type version is generated for the VARRAY type. The same changes can be applied to nested table types.

Options like INVALIDATE and CASCADE are provided to either invalidate all dependent objects or propagate the change to its type and table dependents.

Increasing VARRAY Limit Size

The ALTER TYPE ... MODIFY LIMIT syntax allows increasing the number of elements of a VARRAY type. If the number of elements of the VARRAY type is increased, a new type version is generated for the VARRAY type and this is maintained as part of the history of the type changes.

Options like INVALIDATE and CASCADE are provided to either invalidate all dependent objects or propagate the change to its type and table dependents.

Overview of Type Inheritance

An object type can be created as a subtype of an existing object type. A single inheritance model is supported: the subtype can be derived from only one parent type. A type inherits all the attributes and methods of its direct supertype. It can add new attributes and methods, and it can override any of the inherited methods.

Furthermore, a subtype can itself be refined by defining another subtype under it, thus building up type hierarchies.

Example picture illustrates two subtypes, Student_t and Employee_t, created under Person_t.

<< Part1: Native Datatypes vs. Object Datatypes

Part3: Object Tables, Object Views and REF's >>

Object-Oriented Features of Oracle - Part 1: Native Datatypes vs. Object Datatypes

Bilal Hatipoglu — 13.08.2007 08:22:00

This was actually my presentation on Aug 6, 2007 in Turkcell, about "Object Datatypes and Object Views" but I will write more on this article. This article consists of three parts.

Oracle is a relational database, but it has a great feature of object-oriented abstraction. Hence, you can use Oracle as a fully object-oriented database and develop your applications on this was. Also, you can use this object-oriented features on your relational schema, without modifying the relational data, just using object views.

You can download my presentation about "Object Relational Features of Oracle" from here.

Let's start with native datatypes vs. object datatypes:

NATIVE DATATYPES

1- CHARACTER DATATYPES

The character datatypes store character (alphanumeric) data in strings, with byte values corresponding to the character encoding scheme, generally called a character set or code page.

a) CHAR Datatype

The CHAR datatype stores fixed-length character strings. When you create a table with a CHAR column, you must specify a string length (in bytes or characters) between 1 and 2000 bytes for the CHAR column width. The default is 1 byte. Oracle then guarantees that:

When you insert or update a row in the table, the value for the CHAR column has the fixed length.

If you give a shorter value, then the value is blank-padded to the fixed length.

If a value is too large, Oracle returns an error.

Oracle compares CHAR values using blank-padded comparison semantics.

b) VARCHAR2 and VARCHAR Datatypes

The VARCHAR2 datatype stores variable-length character strings. When you create a table with a VARCHAR2 column, you specify a maximum string length (in bytes or characters) between 1 and 4000 bytes for the VARCHAR2 column. For each row, Oracle stores each value in the column as a variable-length field unless a value exceeds the column's maximum length, in which case Oracle returns an error. Using VARCHAR2 and VARCHAR saves on space used by the table.

For example, assume you declare a column VARCHAR2 with a maximum size of 50 characters. In a single-byte character set, if only 10 characters are given for the VARCHAR2 column value in a particular row, the column in the row's row piece stores only the 10 characters (10 bytes), not 50.

Oracle compares VARCHAR2 values using nonpadded comparison semantics.

The VARCHAR datatype is synonymous with the VARCHAR2 datatype. To avoid possible changes in behavior, always use the VARCHAR2 datatype to store variable-length character strings.

c) NCHAR and NVARCHAR2 Datatypes

NCHAR and NVARCHAR2 are Unicode datatypes that store Unicode character data. The character set of NCHAR and NVARCHAR2 datatypes can only be either AL16UTF16 or UTF8 and is specified at database creation time as the national character set. AL16UTF16 and UTF8 are both Unicode encoding.

The NCHAR datatype stores fixed-length character strings that correspond to the national character set.

The NVARCHAR2 datatype stores variable length character strings.

When you create a table with an NCHAR or NVARCHAR2 column, the maximum size specified is always in character length semantics. Character length semantics is the default and only length semantics for NCHAR or NVARCHAR2.

For example, if national character set is UTF8, then the following statement defines the maximum byte length of 90 bytes:

CREATE TABLE tab1 (col1 NCHAR(30));

This statement creates a column with maximum character length of 30. The maximum byte length is the multiple of the maximum character length and the maximum number of bytes in each character.

NCHAR
The maximum length of an NCHAR column is 2000 bytes. It can hold up to 2000 characters. The actual data is subject to the maximum byte limit of 2000. The two size constraints must be satisfied simultaneously at run time.

NVARCHAR2
The maximum length of an NVARCHAR2 column is 4000 bytes. It can hold up to 4000 characters. The actual data is subject to the maximum byte limit of 4000. The two size constraints must be satisfied simultaneously at run time.

d) LONG Datatype

Columns defined as LONG can store variable-length character data containing up to 2 gigabytes of information. LONG data is text data that is to be appropriately converted when moving among different systems.

LONG datatype columns are used in the data dictionary to store the text of view definitions. You can use LONG columns in SELECT lists, SET clauses of UPDATE statements, and VALUES clauses of INSERT statements.

2- NUMERIC DATATYPES

a) NUMBER Datatype

The NUMBER datatype stores fixed and floating-point numbers. Numbers of virtually any magnitude can be stored and are guaranteed portable among different systems operating Oracle, up to 38 digits of precision.

The following numbers can be stored in a NUMBER column:

• Positive numbers in the range 1 x 10-130 to 9.99...9 x 10125 with up to 38 significant digits
• Negative numbers from -1 x 10-130 to 9.99...99 x 10125 with up to 38 significant digits
• Zero
• Positive and negative infinity (generated only by importing from an Oracle Version 5 database)

b) BINARY_FLOAT Datatype

BINARY_FLOAT is a 32-bit, single-precision floating-point number datatype. Each BINARY_FLOAT value requires 5 bytes, including a length byte.

c) BINARY_DOUBLE Datatype

BINARY_DOUBLE is a 64-bit, double-precision floating-point number datatype. Each BINARY_DOUBLE value requires 9 bytes, including a length byte.

3- DATE Datatype

The DATE datatype stores point-in-time values (dates and times) in a table. The DATE datatype stores the year (including the century), the month, the day, the hours, the minutes, and the seconds (after midnight).

Oracle can store dates in the Julian era, ranging from January 1, 4712 BCE through December 31, 4712 CE (Common Era, or 'AD'). Unless BCE ('BC' in the format mask) is specifically used, CE date entries are the default.

Oracle uses its own internal format to store dates. Date data is stored in fixed-length fields of seven bytes each, corresponding to century, year, month, day, hour, minute, and second.

4- LOB Datatypes

The LOB datatypes BLOB, CLOB, NCLOB, and BFILE enable you to store and manipulate large blocks of unstructured data (such as text, graphic images, video clips, and sound waveforms) in binary or character format. They provide efficient, random, piece-wise access to the data. Oracle recommends that you always use LOB datatypes over LONG datatypes. You can perform parallel queries (but not parallel DML or DDL) on LOB columns.

LOB datatypes differ from LONG and LONG RAW datatypes in several ways. For example:

A table can contain multiple LOB columns but only one LONG column.

A table containing one or more LOB columns can be partitioned, but a table containing a LONG column cannot be partitioned.

The maximum size of a LOB is 8 terabytes, and the maximum size of a LONG is only 2 gigabytes.

LOBs support random access to data, but LONGs support only sequential access.

LOB datatypes (except NCLOB) can be attributes of a user-defined object type but LONG datatypes cannot.

Temporary LOBs that act like local variables can be used to perform transformations on LOB data. Temporary internal LOBs (BLOBs, CLOBs, and NCLOBs) are created in a temporary tablespace and are independent of tables. For LONG datatypes, however, no temporary structures are available.

Tables with LOB columns can be replicated, but tables with LONG columns cannot.

SQL statements define LOB columns in a table and LOB attributes in a user-defined object type. When defining LOBs in a table, you can explicitly specify the tablespace and storage characteristics for each LOB.

a) BLOB Datatype

The BLOB datatype stores unstructured binary data in the database. BLOBs can store up to 8 terabytes of binary data.

BLOBs participate fully in transactions. Changes made to a BLOB value by the DBMS_LOB package, PL/SQL, or the OCI can be committed or rolled back. However, BLOB locators cannot span transactions or sessions.

b) CLOB and NCLOB Datatypes

The CLOB and NCLOB datatypes store up to 8 terabytes of character data in the database. CLOBs store database character set data, and NCLOBs store Unicode national character set data. Storing varying-width LOB data in a fixed-width Unicode character set internally enables Oracle to provide efficient character-based random access on CLOBs and NCLOBs.

c) BFILE Datatype

The BFILE datatype stores unstructured binary data in operating-system files outside the database. A BFILE column or attribute stores a file locator that points to an external file containing the data. BFILEs can store up to 8 terabytes of data.

BFILEs are read only; you cannot modify them. They support only random (not sequential) reads, and they do not participate in transactions. The underlying operating system must maintain the file integrity, security, and durability for BFILEs. The database administrator must ensure that the file exists and that Oracle processes have operating-system read permissions on the file.

5- RAW and LONG RAW Datatypes

The RAW and LONG RAW datatypes are used for data that is not to be interpreted (not converted when moving data between different systems) by Oracle. These datatypes are intended for binary data or byte strings. For example, LONG RAW can be used to store graphics, sound, documents, or arrays of binary data. The interpretation depends on the use.

RAW is a variable-length datatype like the VARCHAR2 character datatype, except Oracle Net Services (which connects user sessions to the instance) and the Import and Export utilities do not perform character conversion when transmitting RAW or LONG RAW data. In contrast, Oracle Net Services and Import/Export automatically convert CHAR, VARCHAR2, and LONG data between the database character set and the user session character set, if the two character sets are different.

When Oracle automatically converts RAW or LONG RAW data to and from CHAR data, the binary data is represented in hexadecimal form with one hexadecimal character representing every four bits of RAW data. For example, one byte of RAW data with bits 11001011 is displayed and entered as 'CB'.

LONG RAW data cannot be indexed, but RAW data can be indexed.

6- ROWID and UROWID Datatypes

Oracle uses a ROWID datatype to store the address (rowid) of every row in the database.

A single datatype called the universal rowid, or UROWID, supports both logical and physical rowids, as well as rowids of foreign tables such as non-Oracle tables accessed through a gateway.

A column of the UROWID datatype can store all kinds of rowids. The value of the COMPATIBLE initialization parameter (for file format compatibility) must be set to 8.1 or higher to use UROWID columns.

7- ANSI, DB2, and SQL/DS Datatypes

SQL statements that create tables and clusters can also use ANSI datatypes and datatypes from IBM's products SQL/DS and DB2. Oracle recognizes the ANSI or IBM datatype name that differs from the Oracle datatype name, records it as the name of the datatype of the column, and then stores the column's data in an Oracle datatype based on the conversions.

8- XML Datatypes

XMLType can be used like any other user-defined type. XMLType can be used as the datatype of columns in tables and views. Variables of XMLType can be used in PL/SQL stored procedures as parameters, return values, and so on. You can also use XMLType in PL/SQL, SQL and Java, and through JDBC and OCI.

A number of useful functions that operate on XML content have been provided. Many of these are provided both as SQL functions and as member functions of XMLType. For example, function extract extracts a specific node(s) from an XMLType instance. You can use XMLType in SQL queries in the same way as any other user-defined datatypes in the system.

9- URI Datatypes

A URI, or uniform resource identifier, is a generalized kind of URL. Like a URL, it can reference any document, and can reference a specific part of a document. It is more general than a URL because it has a powerful mechanism for specifying the relevant part of the document.

OBJECT DATATYPES

Introduction

Object types and other user-defined datatypes let you define datatypes that model the structure and behavior of the data in their applications. An object view is a virtual object table.

Oracle object technology is a layer of abstraction built on Oracle's relational technology. New object types can be created from any built-in database types or any previously created object types, object references, and collection types. Metadata for user-defined types is stored in a schema available to SQL, PL/SQL, Java, and other published interfaces. Object datatypes make it easier to work with complex data, such as images, audio, and video.

An object type differs from native SQL datatypes in that it is user-defined, and it specifies both the underlying persistent data (attributes) and the related behaviors (methods). Object types are abstractions of the real-world entities, for example, purchase orders. Object types store structured business data in its natural form and allow applications to retrieve it that way.

Internally, statements about objects are still basically statements about relational tables and columns, and you can continue to work with relational datatypes and store data in relational tables. But you have the option to take advantage of object-oriented features too. You can use object-oriented features while continuing to work with most of your relational data, or you can go over to an object-oriented approach entirely. For instance, you can define some object datatypes and store the objects in columns in relational tables. You can also create object views of existing relational data to represent and access this data according to an object model. Or you can store object data in object tables, where each row is an object.

Reasons to Use Object Datatypes

• In general, the object-type model is similar to the class mechanism found in C++ and Java. Like classes, objects make it easier to model complex, real-world business entities and logic, and the reusability of objects makes it possible to develop database applications faster and more efficiently. The object type is an excellent way to extend PL/SQL with new functionality in the same way a class structure does this in C++ or Java.

• By natively supporting object types in the database, Oracle enables application developers to directly access the data structures used by their applications. No mapping layer is required between client-side objects and the relational database columns and tables that contain the data.

• Object abstraction and the encapsulation of object behaviors also make applications easier to understand and maintain.

• Object types can be used to enforce standardization as well. I can create a new type, say ADDRESS_TYPE, which encapsulates the definition of an address - the discrete components that make it up. I can even add convenience functions (methods) around this type, perhaps to return the address in a format suitable for printing on labels for example. Now, whenever I create a table needing a column that is an address, I can simply declare it is as ADDRESS_TYPE. The attributes that constitute an address will be added to my table for me automatically.

• Object types can be used to present an object relational view of strictly relational data: Object Views

Objects Can Encapsulate Operations Along with Data

Database tables contain only data. Objects can include the ability to perform operations that are likely to be needed on that data. Thus a purchase order object might include a method to sum the cost of all the items purchased. Or a customer object might have methods to return the customer's buying history and payment pattern. An application can simply call the methods to retrieve the information.

Objects Are Efficient

Using object types makes for greater efficiency:

• Object types and their methods are stored with the data in the database, so they are available for any application to use. Developers can benefit from work that is already done and do not need to re-create similar structures in every application.

• You can fetch and manipulate a set of related objects as a single unit. A single request to fetch an object from the server can retrieve other objects that are connected to it. For example, when you select a customer object and get the customer's name, phone, and the multiple parts of his address in a single round-trip between the client and the server. When you reference a column of a SQL object type, you retrieve the whole object.

Objects Can Represent Part-Whole Relationships

In a relational system, it is awkward to represent complex part-whole relationships. A piston and an engine have the same status in a table for stock items. To represent pistons as parts of engines, you must create complicated schemas of multiple tables with primary key-foreign key relationships. Object types, on the other hand, give you a rich vocabulary for describing part-whole relationships. An object can have other objects as attributes, and the attribute objects can have their own object attributes too. An entire parts-list hierarchy can be built up in this way from interlocking object types.

Part2: Object Datatypes and Collection Datatypes >>

Difference between db block gets and consistent gets

Bilal Hatipoglu — 12.08.2007 14:27:36

Quoted from OraFAQ Mail list:

Q: What is db block gets and consistent gets? How can I reduce consistent gets? Ask Tom says each consistent gets is latch, how it could be?

A: A 'db block get' is a current mode get. That is, it's the most up-to-date copy of the data in that block, as it is right now, or currently. There can only be one current copy of a block in the buffer cache at any time. Db block gets generally are used when DML changes data in the database. In that case, row-level locks are implicitly taken on the updated rows. There is also at least one well-known case where a select statement does a db block get, and does not take a lock. That is, when it does a full table scan or fast full index scan, Oracle will read the segment header in current mode (multiple times, the number varies based on Oracle version).

A 'consistent get' is when Oracle gets the data in a block which is consistent with a given point in time, or SCN. The consistent get is at the heart of Oracle's read consistency mechanism. When blocks are fetched in order to satisfy a query result set, they are fetched in consistent mode. If no block in the buffer cache is consistent to the correct point in time, Oracle will (attempt to) reconstruct that block using the information in the rollback segments. If it fails to do so, that's when a query errors out with the much dreaded, much feared, and much misunderstood ORA-1555 "snapshot too old".

As to latching, and how it relates, well, consider that the block buffers are in the SGA, which is shared memory. To avoid corruption, latches are used to serialize access to many linked lists and data structures that point to the buffers as well as the buffers themselves. It is safe to say that each consistent get introduces serialization to the system, and by tuning SQL to use more efficient access paths, you can get the same answer to the same query but do less consistent gets. This not only consumes less CPU, it also can significantly reduce latching which reduces serialization and makes your system more scalable.

Feed users, click here to go to this topic's page. You can find related topics and reference links and you can read the comments for this topic

Feed users, click here to post a comment for this topic

Note: There might be problems about HTML code and parsing and displaying HTML code of your viewer application. Also, there are problems caused by Google Reader on displaying of code samples on the entry. Also, the entry might be updated. So, it is recommended to visit the page of the entry.

Turkcell Staj Günlüğü - 11: Autonomous Transactions ve Dynamic SQL

Bilal Hatipoglu — 12.08.2007 11:07:30

Merhaba, bu makalede 24.07.2007'de Hakkı Oktay'ın yaptığı Autonomous Transactions ve Dynamic SQL sunuma göz atacağız. Hakkı abinin sunumu gerçekten güzeldi, her developer'ın bilmesi gereken bir konu Dynamic SQL. Autonomous Transaction konsepti de önemli.

Hakkı Oktay'ın "Autonomous Transactions ve Dynamic SQL" sunumunu buradan indirebilirsiniz.

• Autonomous Transaction'ları Mert İNAN'ın sunumunda işlemiştik. Güzel bir örneğiyle birlikte bu makaleden ulaşabilirsiniz.

• Dynamic SQL, runtime'de oluşturulan bir SQL'in çalıştırılmasıdır. Normalde programlarda veya PL/SQL bloklarında sorgu yazarken hard code olarak yazıp çalıştırılan SQL'ler static SQL'dir. Örneğin SQL*Plus üzerine yazdığımız bütün SQL'leri dynamic SQL ile çalıştırır. Dynamic SQL kavramı kullanım örneklerinden sonra daha anlaşılır olacaktır.

• Dynamic SQL esnektir, kullanımı büyük esneklik sağlar, Ama Static SQL kullanmak çoğu zaman daha performanslı olacağı için static SQL ile yapılabilecek şeyleri static SQL ile yapmalı, dynamic SQL'i mecbur olduğunda kullanmalıdır.

• Dynamic SQL'i ne zaman kullanırız? Run-time'da sorgularımızın inputlara gre veya başka şartlarda değişebileceği durumlarda veya DDL çalıştırmak için kullanırız. (Normalde PL/SQL bloğu içinde DDL çalıştırılamıyor)

• Dynamic SQL, PL/SQL içinde iki şekilde kullanılabilir: Birincisi EXECUTE IMMEDIATE komutu ile, ikincisi ise DBMS_SQL paketi ile. Execute Immediate ile DML'leri, DDL'leri, PL/SQL bloklarını, Transaction Control komutlarını ve Session Control komutlarını kolaylıkla çalıştırabilirsiniz. Bu komut, sorgunun dinamik olarak runtime'da parse edilip çalıştırılmasını sağlar. DBMS_SQL paketi ise, biraz daha gelişmiş özellikleri barındırır. Daha çok özel amaçlarla kullanılır. Örneğin, sorgudan dönen kolon isimleri veya kolon sayısı gibi bir bilgi lazım olunca DBMS_SQL paketi kullanılabilir. Fakat daha fazla özellik barındırmayısla birlikte, kullanımı biraz daha zordur ve biraz daha yavaş çalışır. DBMS_SQL paketinin kullanımı ile ilgili ayrıntılı bilgiyi referanslarda da bulabilirsiniz. Her iki kullanım örneklerini de sunumda bulabilirsiniz.

• Native Dynamic SQL(EXEC. IMMED.)'in avantajları:

1- Kullanımı kolaydır.
2- Daha hızlı çalışır.
3- Kullanıcı tanımlı tipleri destekler.
4- Dönen kayıtları RECORD'lar içine atabilir.

• DBMS_SQL paketinin avantajları:

1- Input ve output sayısı bilinmediğinde de kullanılabilir.
2- Select'ten dönecek kolon sayısı veya isimleri belli olmadığında da çalıştırılabilir.
3- 32KB'dan daha uzun sorguları da çalıştırabilir.
4- Tekrar kullanılabilirliği sağlar.

• Amacımız genellikle performans ve kolaylık avantajını yaklamak olduğu için, yapabildiğimiz herşeyi EXEC. IMMED. ile yapmamız daa mantıklı. Ama DBMS_SQL paketine gerçekten ihtiyacımız olan durumlr da oluyor, bu yüzden bu paketi ve yetkinliklerini bilmemiz de önemli.

Bu kısa, ama önemli ve zevkli sunumun notları da bu kadar. Daha fazla bilgi edinmek isteyenler referans linkleri kullanabilirler. Herkese iyi çalışmalar.

Feed users, click here to go to this topic's page. You can find related topics and reference links and you can read the comments for this topic

Feed users, click here to post a comment for this topic

Note: There might be problems about HTML code and parsing and displaying HTML code of your viewer application. Also, there are problems caused by Google Reader on displaying of code samples on the entry. Also, the entry might be updated. So, it is recommended to visit the page of the entry.

Turkcell Staj Günlüğü - 10: Import, Export ve SQL Loader

Bilal Hatipoglu — 11.08.2007 18:46:54

Merhaba, uzun bir aradan sonra yazı dizisine devam ediyorum :) Turkcell'de işler yoğunlaştı,stajyer hiç stajyer gibi olmadığı için, hepimiz gerçek projelerde çalışmaya devam ediyoruz, bu aralar da benim üzerinde çalıştığım proje için yoğundu o yüzden fazla birşey yazamadım. Ama haftasonunu kendime ayırıp 5-6 giriş yazmayı düşünüyorum, ha bu durumda kend me mi bilgisayarıma mı ayırmış olacağım haftasonunu o tartışılır ama zaten bilgisayarlarla bütünleştiğim için çok birşey değişmeyecektir :)

Bu makalede, 23.07.2007 ve 24.07.2007'de 2 ayrı sunum yapmış olan Hakkı Oktay'ın ilk sunumu üzerinden geçeceğiz. İlk konumuz Export, Import ve SQL Loader.

Her zamanki gibi, sunumun orjinalini buradan indirebilirsiniz. Ben de ufak notlarımı ve birkaç örneği paylaşacağım:

• Import(IMP) ve Export(EXP), Oracle'ın iki aracının adıdır. Oracle'ın en eski araçlarındandır. Adlarından da anlaşılabileceği gibi, database'deki çeşitli veriyi(ilerde değineceğiz) dosya olarak kaydedebilmeyi, ve sonra yine bunu geri yükleyebilmeyi sağlıyorlar.

• IMP ve EXP ile database'deki tabloları, schema'ları veya bütün database'i import veya export edebilirsiniz.

• Sunumda geçen DataPump Export ve DataPump Import bu tool'ların uzun isimleri, farklı birşey olarak algılamayın yani :)

• IMP ve EXP sadece data'yı değil, metadata'yı da import veya export edebilir. Yani içerideki data'yla alakalı bilgiyi de saklar, kolonun adı nedir, veri tipi nedir gibi. Zaten bunlar saklanmazsa data'yı düzgün bir şekilde import etmek ya mümkün olmaz veya çok zahmetli olur.

• Export'un çıkardığı dump dosyası binary bir dosyadır. Text editor'lerle içini açıp veriyi göremez, değiştiremeyiz. Bu dosyayı sadece Import(IMP) aracı okuyup veriyi yükleyebilir.

• Esas bilinesi gereken önemli özellik, çıkan dump dosyasının platformlar arasında taşınabilir olmasıdır. Linux'daki bir Oracle database'den çıkarılan dump dosyası, Windows'dakine kolayca aktarılıp kullanılabilir.

Import ve Export'un çeşitli mod'ları ve çeşitli filtreleme opsiyonları var, ayrıntıları sunumda ve dökümantasyonda bulabilirsiniz.

Export ile ilgili ufak bir örnek:

SQL*Plus: Release 10.2.0.1.0 - Production on Sat Aug 11 17:24:00 2007

Copyright (c) 1982, 2005, Oracle.  All rights reserved.

Enter password:

Connected to:
Oracle Database 10g Enterprise Edition Release 10.2.0.1.0 - Production
With the Partitioning, OLAP and Data Mining options

SQL> drop user dtest cascade;

User dropped.

SQL> create user dtest identified by test;

User created.

SQL> grant connect, resource to dtest;

Grant succeeded.

SQL> drop directory dpdata1;

Directory dropped.

SQL> create directory dpdata1 as '/home/oracle';

Directory created.

SQL> grant read, write on directory dpdata1 to dtest;

Grant succeeded.

SQL> grant EXP_FULL_DATABASE to  dtest;

Grant succeeded.

SQL> grant IMP_FULL_DATABASE to dtest;

Grant succeeded.

SQL> SELECT directory_path FROM dba_directories WHERE directory_name = 'DPDATA1';

DIRECTORY_PATH
----------------------------------------------------------------------------------------
/home/oracle

SQL>conn dtest/test@bilalEE
Connected.

SQL>create table ttest nologging as select object_id, object_name, object_type
from all_objects; 

Table created.

SQL> quit
Disconnected from Oracle Database 10g Enterprise Edition Release 10.2.0.1.0 - Production
With the Partitioning, OLAP and Data Mining options

C:\>expdp dtest/test DIRECTORY=dpdata1 DUMPFILE=ttest.dmp

Export: Release 10.2.0.1.0 - Production on Saturday, 11 August, 2007 17:15:28

Copyright (c) 2003, 2005, Oracle.  All rights reserved.

Connected to: Oracle Database 10g Enterprise Edition Release 10.2.0.1.0 - Production
With the Partitioning, OLAP and Data Mining options
Starting "DTEST"."SYS_EXPORT_SCHEMA_01":  dtest/******** DIRECTORY=dpdata1
DUMPFILE=ttest.dmp 
Estimate in progress using BLOCKS method...
Processing object type SCHEMA_EXPORT/TABLE/TABLE_DATA
Total estimation using BLOCKS method: 3 MB
Processing object type SCHEMA_EXPORT/USER
Processing object type SCHEMA_EXPORT/SYSTEM_GRANT
Processing object type SCHEMA_EXPORT/ROLE_GRANT
Processing object type SCHEMA_EXPORT/DEFAULT_ROLE
Processing object type SCHEMA_EXPORT/PRE_SCHEMA/PROCACT_SCHEMA
Processing object type SCHEMA_EXPORT/TABLE/TABLE
Processing object type SCHEMA_EXPORT/TABLE/INDEX/INDEX
Processing object type SCHEMA_EXPORT/TABLE/CONSTRAINT/CONSTRAINT
Processing object type SCHEMA_EXPORT/TABLE/INDEX/STATISTICS/INDEX_STATISTICS
Processing object type SCHEMA_EXPORT/TABLE/COMMENT
. . exported "DTEST"."TTEST"                             1.984 MB   49721 rows
Master table "DTEST"."SYS_EXPORT_SCHEMA_01" successfully loaded/unloaded
******************************************************************************
Dump file set for DTEST.SYS_EXPORT_SCHEMA_01 is:
  /home/oracle/ttest.dmp
Job "DTEST"."SYS_EXPORT_SCHEMA_01" successfully completed at 17:16:53

C:\>

Download Code

Import ve Export'un help'i de şu şekilde:

C:\>expdp help=y

Export: Release 10.2.0.1.0 - Production on Saturday, 11 August, 2007 17:39:01

Copyright (c) 2003, 2005, Oracle.  All rights reserved.


The Data Pump export utility provides a mechanism for transferring data objects
between Oracle databases. The utility is invoked with the following command:

   Example: expdp scott/tiger DIRECTORY=dmpdir DUMPFILE=scott.dmp

You can control how Export runs by entering the 'expdp' command followed
by various parameters. To specify parameters, you use keywords:

   Format:  expdp KEYWORD=value or KEYWORD=(value1,value2,...,valueN)
   Example: expdp scott/tiger DUMPFILE=scott.dmp DIRECTORY=dmpdir SCHEMAS=scott
               or TABLES=(T1:P1,T1:P2), if T1 is partitioned table

USERID must be the first parameter on the command line.

Keyword               Description (Default)
------------------------------------------------------------------------------
ATTACH                Attach to existing job, e.g. ATTACH [=job name].
COMPRESSION           Reduce size of dumpfile contents where valid
                      keyword values are: (METADATA_ONLY) and NONE.
CONTENT               Specifies data to unload where the valid keywords are:
                      (ALL), DATA_ONLY, and METADATA_ONLY.
DIRECTORY             Directory object to be used for dumpfiles and logfiles.
DUMPFILE              List of destination dump files (expdat.dmp),
                      e.g. DUMPFILE=scott1.dmp, scott2.dmp, dmpdir:scott3.dmp.
ENCRYPTION_PASSWORD   Password key for creating encrypted column data.
ESTIMATE              Calculate job estimates where the valid keywords are:
                      (BLOCKS) and STATISTICS.
ESTIMATE_ONLY         Calculate job estimates without performing the export.
EXCLUDE               Exclude specific object types, e.g. EXCLUDE=TABLE:EMP.
FILESIZE              Specify the size of each dumpfile in units of bytes.
FLASHBACK_SCN         SCN used to set session snapshot back to.
FLASHBACK_TIME        Time used to get the SCN closest to the specified time.
FULL                  Export entire database (N).
HELP                  Display Help messages (N).
INCLUDE               Include specific object types, e.g. INCLUDE=TABLE_DATA.
JOB_NAME              Name of export job to create.
LOGFILE               Log file name (export.log).
NETWORK_LINK          Name of remote database link to the source system.
NOLOGFILE             Do not write logfile (N).
PARALLEL              Change the number of active workers for current job.
PARFILE               Specify parameter file.
QUERY                 Predicate clause used to export a subset of a table.
SAMPLE                Percentage of data to be exported;
SCHEMAS               List of schemas to export (login schema).
STATUS                Frequency (secs) job status is to be monitored where
                      the default (0) will show new status when available.
TABLES                Identifies a list of tables to export - one schema only.
TABLESPACES           Identifies a list of tablespaces to export.
TRANSPORT_FULL_CHECK  Verify storage segments of all tables (N).
TRANSPORT_TABLESPACES List of tablespaces from which metadata will be unloaded.
VERSION               Version of objects to export where valid keywords are:
                      (COMPATIBLE), LATEST, or any valid database version.

The following commands are valid while in interactive mode.
Note: abbreviations are allowed

Command               Description
------------------------------------------------------------------------------
ADD_FILE              Add dumpfile to dumpfile set.
CONTINUE_CLIENT       Return to logging mode. Job will be re-started if idle.
EXIT_CLIENT           Quit client session and leave job running.
FILESIZE              Default filesize (bytes) for subsequent ADD_FILE commands.
HELP                  Summarize interactive commands.
KILL_JOB              Detach and delete job.
PARALLEL              Change the number of active workers for current job.
                      PARALLEL=.
START_JOB             Start/resume current job.
STATUS                Frequency (secs) job status is to be monitored where
                      the default (0) will show new status when available.
                      STATUS[=interval]
STOP_JOB              Orderly shutdown of job execution and exits the client.
                      STOP_JOB=IMMEDIATE performs an immediate shutdown of the
                      Data Pump job.


C:\>impdp help=y

Import: Release 10.2.0.1.0 - Production on Saturday, 11 August, 2007 17:39:56

Copyright (c) 2003, 2005, Oracle.  All rights reserved.


The Data Pump Import utility provides a mechanism for transferring data objects
between Oracle databases. The utility is invoked with the following command:

     Example: impdp scott/tiger DIRECTORY=dmpdir DUMPFILE=scott.dmp

You can control how Import runs by entering the 'impdp' command followed
by various parameters. To specify parameters, you use keywords:

     Format:  impdp KEYWORD=value or KEYWORD=(value1,value2,...,valueN)
     Example: impdp scott/tiger DIRECTORY=dmpdir DUMPFILE=scott.dmp

USERID must be the first parameter on the command line.

Keyword               Description (Default)
------------------------------------------------------------------------------
ATTACH                Attach to existing job, e.g. ATTACH [=job name].
CONTENT               Specifies data to load where the valid keywords are:
                      (ALL), DATA_ONLY, and METADATA_ONLY.
DIRECTORY             Directory object to be used for dump, log, and sql files.
DUMPFILE              List of dumpfiles to import from (expdat.dmp),
                      e.g. DUMPFILE=scott1.dmp, scott2.dmp, dmpdir:scott3.dmp.
ENCRYPTION_PASSWORD   Password key for accessing encrypted column data.
                      This parameter is not valid for network import jobs.
ESTIMATE              Calculate job estimates where the valid keywords are:
                      (BLOCKS) and STATISTICS.
EXCLUDE               Exclude specific object types, e.g. EXCLUDE=TABLE:EMP.
FLASHBACK_SCN         SCN used to set session snapshot back to.
FLASHBACK_TIME        Time used to get the SCN closest to the specified time.
FULL                  Import everything from source (Y).
HELP                  Display help messages (N).
INCLUDE               Include specific object types, e.g. INCLUDE=TABLE_DATA.
JOB_NAME              Name of import job to create.
LOGFILE               Log file name (import.log).
NETWORK_LINK          Name of remote database link to the source system.
NOLOGFILE             Do not write logfile.
PARALLEL              Change the number of active workers for current job.
PARFILE               Specify parameter file.
QUERY                 Predicate clause used to import a subset of a table.
REMAP_DATAFILE        Redefine datafile references in all DDL statements.
REMAP_SCHEMA          Objects from one schema are loaded into another schema.
REMAP_TABLESPACE      Tablespace object are remapped to another tablespace.
REUSE_DATAFILES       Tablespace will be initialized if it already exists (N).
SCHEMAS               List of schemas to import.
SKIP_UNUSABLE_INDEXES Skip indexes that were set to the Index Unusable state.
SQLFILE               Write all the SQL DDL to a specified file.
STATUS                Frequency (secs) job status is to be monitored where
                      the default (0) will show new status when available.
STREAMS_CONFIGURATION Enable the loading of Streams metadata
TABLE_EXISTS_ACTION   Action to take if imported object already exists.
                      Valid keywords: (SKIP), APPEND, REPLACE and TRUNCATE.
TABLES                Identifies a list of tables to import.
TABLESPACES           Identifies a list of tablespaces to import.
TRANSFORM             Metadata transform to apply to applicable objects.
                      Valid transform keywords: SEGMENT_ATTRIBUTES, STORAGE
                      OID, and PCTSPACE.
TRANSPORT_DATAFILES   List of datafiles to be imported by transportable mode.
TRANSPORT_FULL_CHECK  Verify storage segments of all tables (N).
TRANSPORT_TABLESPACES List of tablespaces from which metadata will be loaded.
                      Only valid in NETWORK_LINK mode import operations.
VERSION               Version of objects to export where valid keywords are:
                      (COMPATIBLE), LATEST, or any valid database version.
                      Only valid for NETWORK_LINK and SQLFILE.

The following commands are valid while in interactive mode.
Note: abbreviations are allowed

Command               Description (Default)
------------------------------------------------------------------------------
CONTINUE_CLIENT       Return to logging mode. Job will be re-started if idle.
EXIT_CLIENT           Quit client session and leave job running.
HELP                  Summarize interactive commands.
KILL_JOB              Detach and delete job.
PARALLEL              Change the number of active workers for current job.
                      PARALLEL=.
START_JOB             Start/resume current job.
                      START_JOB=SKIP_CURRENT will start the job after skipping
                      any action which was in progress when job was stopped.
STATUS                Frequency (secs) job status is to be monitored where
                      the default (0) will show new status when available.
                      STATUS[=interval]
STOP_JOB              Orderly shutdown of job execution and exits the client.
                      STOP_JOB=IMMEDIATE performs an immediate shutdown of the
                      Data Pump job.


C:\>

• SQL Loader da, Oracle'ın yeni nesil data import aracıdır. Oldukça esnek özellikleri vardır. Bu daha çok farklı database'lerden Oracle'a veri import etmek için kullanılır. Data'yı filtreleme ve load ederken modifiye etme gibi seçenekleri de vardır.

• SQL Loader input olarak bir data dosyası, birde o data hakkında bilgi içeren kontrol dosyası alır. Örnek data dosyası ve kontrol dosyalarını sunumda bulabilirsiniz.

Feed users, click here to go to this topic's page. You can find related topics and reference links and you can read the comments for this topic

Feed users, click here to post a comment for this topic

Note: There might be problems about HTML code and parsing and displaying HTML code of your viewer application. Also, there are problems caused by Google Reader on displaying of code samples on the entry. Also, the entry might be updated. So, it is recommended to visit the page of the entry.

When the explanation doesn't sound quite right...

Bilal Hatipoglu — 03.08.2007 08:03:49

Quoted from Mr. KYTE's blog:

Q: Under what conditions, autotrace & explain plan can not give the correct execution plan of a sql?

A: To start with the answer to this - we need to understand that autotrace is just a feature of SQL Plus that automates an explain plan for us - so, autotrace and explain plan are sort of synonymous in this regard. I'll be using Oracle 10g Release 2 in these examples and will be using autotrace or sql_trace=true and TKPROF - you can get the same results in 9i and later using EXPLAIN PLAN and DBMS_XPLAN.DISPLAY to see the results.

Explain Plan is in the here and now...

This is the first problem with explain plan - it is in the "here and now". It uses the current optimizer environment, the current set of statistics and so on. That means the explain plan you see in a tkprof could differ from the REAL PLAN used 5 minutes ago (when performance was 'bad').

...

Explain plan is blind to the bind

Explain plan does not "bind peek".

...

Explain plan doesn't see your datatype...

The last bit about explain plan I'll look at is the fact that explain plan doesn't see your bind datatype. It presumes all binds are varchar2's regardless of how the developer is binding.

...

http://tkyte.blogspot.com/2007/04/when-explanation-doesn-sound-quite.html

Feed users, click here to go to this topic's page. You can find related topics and reference links and you can read the comments for this topic

Feed users, click here to post a comment for this topic

Note: There might be problems about HTML code and parsing and displaying HTML code of your viewer application. Also, there are problems caused by Google Reader on displaying of code samples on the entry. Also, the entry might be updated. So, it is recommended to visit the page of the entry.

Bind is bad :) - An interesting case of bind variables fails

Bilal Hatipoglu — 01.08.2007 10:18:14

You’ll hear it from many people, the biggest problem in many applications is the non-use of bind variables. Oracle bind variables are a super important way to make Oracle SQL reentrant.

Why is the use of bind variables such an issue? Well, Oracle uses a signature generation algorithm to assign a hash value to each SQL statement based on the text of the SQL statement. Any change in a statement (including whitespaces or lower-uppercase change) will result in a new hash and thus Oracle assumes it is a new statement. Each new statement must be verified, parsed and have an execution plan generated and stored, all high overhead that might be avoided by using bind variables.

Bind variables does well on most of the cases, but there are some rare cases that can reduce performance. Remember, the main idea on bind variables is to preserve the sql text, so, get the same hash value, so, skip parsing, so, use the same execution plan for the *same* (or it's better to say "similar", but neither the same, nor similar, let's invent a new word: "sameilar" :) ) queries. But there are some cases, especially for tables that has large clustering factor, or the data is not homogenous, or for partitioned tables, the optimizer will generate different execution plans for that "sameilar" queries. So with using bind variables, we force the optimizer to use the same execution plan, usually.

Here, for this article, I just have a scenario which the bind variables fails.

Here is the code:

SQL> drop table bind_is_bad;

Table dropped.

SQL> create table bind_is_bad (id number, name varchar2(200), surname varchar2(200));

Table created.

SQL>
SQL> declare
  2         ran varchar2(200);
  3  begin
  4       for i in 1..100000
  5       loop
  6           ran:=dbms_random.string('l',200);
  7           insert into bind_is_bad values(i,ran,ran);
  8           insert into bind_is_bad values(100001,ran,ran);
  9       end loop;
 10  end;
 11  /

PL/SQL procedure successfully completed.

SQL>
SQL> create index idx_bind_is_bad_id on bind_is_bad(id,name);

Index created.

We just created a new table called "bind_is_bad" and fill the table with id's of 1 to 100000 and randomly generated strings, plus 100000 times the record of the id=100001. Also, we insert them in nested order (1,100001,2,100001,3,100001 ...) to have a great clustering factor value. Then, we have id values of 1..100000 once and 100001 for 100000 times :) Then we created an index on it.

So, let's look at the statistics:

SQL> select table_name,num_rows,blocks,empty_blocks,last_analyzed
from all_tables where table_name='BIND_IS_BAD';

TABLE_NAME                       NUM_ROWS     BLOCKS EMPTY_BLOCKS LAST_ANAL
------------------------------ ---------- ---------- ------------ ---------
BIND_IS_BAD

SQL> select index_name, table_name, blevel, leaf_blocks,
clustering_factor, num_rows, last_analyzed
from all_indexes where index_name='IDX_BIND_IS_BAD_ID';

INDEX_NAME                     TABLE_NAME                         BLEVEL LEAF_BLOCKS
------------------------------ ------------------------------ ---------- -----------
CLUSTERING_FACTOR   NUM_ROWS LAST_ANAL
----------------- ---------- ---------
IDX_BIND_IS_BAD_ID             BIND_IS_BAD                             2        6241
           111757     200000 28-JUL-07

We see there is no statistics about table, and *some* for index. Clustering factor=111757 is really bad for a 200000-row table :)

Now, let's examine the executing plan's for different values of the same query (which are actually the "sameilar" queries :) )

SQL> set autotrace traceonly explain;
SQL> select * from bind_is_bad where id=1;

Execution Plan
----------------------------------------------------------
Plan hash value: 1198113483

---------------------------------------------------------------------------------------
| Id  | Operation                   | Name               | Rows  | Bytes | Cost (%CPU)|
---------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT            |                    |     1 |   217 |     4   (0)|
|   1 |  TABLE ACCESS BY INDEX ROWID| BIND_IS_BAD        |     1 |   217 |     4   (0)|
|*  2 |   INDEX RANGE SCAN          | IDX_BIND_IS_BAD_ID |     1 |       |     3   (0)|
---------------------------------------------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------

   2 - access("ID"=1)

Note
-----
   - dynamic sampling used for this statement

SQL> select * from bind_is_bad where id=2;

Execution Plan
----------------------------------------------------------
Plan hash value: 1198113483

---------------------------------------------------------------------------------------
| Id  | Operation                   | Name               | Rows  | Bytes | Cost (%CPU)|
---------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT            |                    |     1 |   217 |     4   (0)|
|   1 |  TABLE ACCESS BY INDEX ROWID| BIND_IS_BAD        |     1 |   217 |     4   (0)|
|*  2 |   INDEX RANGE SCAN          | IDX_BIND_IS_BAD_ID |     1 |       |     3   (0)|
---------------------------------------------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------

   2 - access("ID"=2)

Note
-----
   - dynamic sampling used for this statement

SQL> select * from bind_is_bad where id=100000;

Execution Plan
----------------------------------------------------------
Plan hash value: 1198113483

---------------------------------------------------------------------------------------
| Id  | Operation                   | Name               | Rows  | Bytes | Cost (%CPU)|
---------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT            |                    |     1 |   217 |     4   (0)|
|   1 |  TABLE ACCESS BY INDEX ROWID| BIND_IS_BAD        |     1 |   217 |     4   (0)|
|*  2 |   INDEX RANGE SCAN          | IDX_BIND_IS_BAD_ID |     1 |       |     3   (0)|
---------------------------------------------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------

   2 - access("ID"=100000)

Note
-----
   - dynamic sampling used for this statement

SQL> select * from bind_is_bad where id=100001;

Execution Plan
----------------------------------------------------------
Plan hash value: 4063739968

---------------------------------------------------------------------------------
| Id  | Operation         | Name        | Rows  | Bytes | Cost (%CPU)| Time     |
---------------------------------------------------------------------------------
|   0 | SELECT STATEMENT  |             | 97122 |    20M|  2619   (1)| 00:00:32 |
|*  1 |  TABLE ACCESS FULL| BIND_IS_BAD | 97122 |    20M|  2619   (1)| 00:00:32 |
---------------------------------------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------

   1 - filter("ID"=100001)

Note
-----
   - dynamic sampling used for this statement

SQL> select /*+ INDEX(bind_is_bad idx_bind_is_bad_id) */ * from bind_is_bad
where id=100001;

Execution Plan
----------------------------------------------------------
Plan hash value: 1198113483

---------------------------------------------------------------------------------------
| Id  | Operation                   | Name               | Rows  | Bytes | Cost (%CPU)|
---------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT            |                    | 97122 |    20M| 59708   (1)|
|   1 |  TABLE ACCESS BY INDEX ROWID| BIND_IS_BAD        | 97122 |    20M| 59708   (1)|
|*  2 |   INDEX RANGE SCAN          | IDX_BIND_IS_BAD_ID | 97122 |       |  3221   (1)|
---------------------------------------------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------

   2 - access("ID"=100001)

Note
-----
   - dynamic sampling used for this statement

SQL>

The plans are as expected, CBO uses the index for 1,2,3 and 100000, but full table scan for 100001. That is actually the best performance way. The decision is so, because of dynamic sampling. The optimizer sees most of the values are 100001, so use full table scan for 100001. We also forced the optimizer to use index scan for 100001 and the difference between FTS(full table scan) and IRS(index range scan) is really giant.

But now, the story begins. When we use bind variables, we save for the cost of parsing and use the same (actually the first parsed) execution plan for this "sameilar" queries, but we map the optimizer's choice for the value 100001 and that will also executed with the others' execution plan(using index).

SQL> set timing on
SQL> declare
  2       TYPE test1_tab IS TABLE OF bind_is_bad%ROWTYPE;
  3       t_tab  test1_tab;
  4  begin
  5       for i in 99999 .. 100001
  6       loop
  7           execute immediate 'select * from bind_is_bad where id=:a'
              bulk collect into t_tab using i;
  8       end loop;
  9  end;
 10  /

PL/SQL procedure successfully completed.

Elapsed: 00:05:03.62

The code with bind variables lasts so much, isn't it? It was just 3 simple queries in a loop! But if we don't use bind variables:

SQL> alter system flush buffer_cache;

System altered.

Elapsed: 00:00:00.26
SQL> declare
  2       TYPE test1_tab IS TABLE OF bind_is_bad%ROWTYPE;
  3       t_tab  test1_tab;
  4  begin
  5       for i in 99999 .. 100001
  6       loop
  7           execute immediate 'select * from bind_is_bad where id='||i
              bulk collect into t_tab;
  8       end loop;
  9  end;
 10  /

PL/SQL procedure successfully completed.

Elapsed: 00:00:07.18

Download the complete pure code suitable for just copy/paste and execute, from here.

You see the drama? Bind variable really fails for a stupid query that just executed 3 times :))

CONCLUSION

This is just a scenario to show you nothing is *always* best to use for all cases. Even the bind variables has some worse cases. So, know the tools, know how to use the tools, but not copy & paste the ideas, just plan your own situation, your own solution for your specific problem. And, don't guess, don't estimate, just test different solutions and different approaches. That is the developers' responsibility to write an efficient code, not the DBAs', to optimize the code block, package or query itself.

Feed users, click here to go to this topic's page. You can find related topics and reference links and you can read the comments for this topic

Feed users, click here to post a comment for this topic

Note: There might be problems about HTML code and parsing and displaying HTML code of your viewer application. Also, there are problems caused by Google Reader on displaying of code samples on the entry. Also, the entry might be updated. So, it is recommended to visit the page of the entry.

Affect of gathering table stats to decision of CBO

Bilal Hatipoglu — 31.07.2007 18:49:30

Optimizer statistics are a collection of data that describe more details about the database and the objects in the database. These statistics are used by the query optimizer to choose the best execution plan for each SQL statement.

The optimizer statistics are stored in the data dictionary. They can be viewed using data dictionary views.

Optimizer statistics include the following:

Table statistics:
• Number of rows
• Number of blocks
• Average row length

Column statistics:
• Number of distinct values (NDV) in column
• Number of nulls in column
• Data distribution (histogram)

Index statistics:
• Number of leaf blocks
• Levels
• Clustering factor

System statistics:
• I/O performance and utilization
• CPU performance and utilization

Because the objects in a database can be constantly changing, statistics must be regularly updated so that they accurately describe these database objects. Statistics are maintained automatically by Oracle or you can maintain the optimizer statistics manually using the DBMS_STATS package.

You can use the DBMS_STATS package or the ANALYZE statement to gather statistics about the physical storage characteristics of a table, index, or cluster. These statistics are stored in the data dictionary and can be used by the optimizer to choose the most efficient execution plan for SQL statements accessing analyzed objects.

Oracle recommends using the more versatile DBMS_STATS package for gathering optimizer statistics, but you must use the ANALYZE statement to collect statistics unrelated to the optimizer, such as empty blocks, average space, and so forth.

The DBMS_STATS package allows both the gathering of statistics, including utilizing parallel execution, and the external manipulation of statistics. Statistics can be stored in tables outside of the data dictionary, where they can be manipulated without affecting the optimizer. Statistics can be copied between databases or backup copies can be made.

AUTOMATIC STATISTICS GATHERING

The recommended approach to gathering statistics is to allow Oracle to automatically gather the statistics. Oracle gathers statistics on all database objects automatically and maintains those statistics in a regularly-scheduled maintenance job. Automated statistics collection eliminates many of the manual tasks associated with managing the query optimizer, and significantly reduces the chances of getting poor execution plans because of missing or stale statistics.

WHEN TO USE MANUAL STATISTICS

Automatic statistics gathering should be sufficient for most database objects which are being modified at a moderate speed. However, there are cases where automatic statistics gathering may not be adequate. Because the automatic statistics gathering runs during an overnight batch window, the statistics on tables which are significantly modified during the day may become stale. There are typically two types of such objects:

• Volatile tables that are being deleted or truncated and rebuilt during the course of the day. • Objects which are the target of large bulk loads which add 10% or more to the object's total size.

MANUAL STATISTICS GATHERING

If you choose not to use automatic statistics gathering, then you need to manually collect statistics in all schemas, including system schemas. If the data in your database changes regularly, you also need to gather statistics regularly to ensure that the statistics accurately represent characteristics of your database objects.

The DBMS_STATS package can gather statistics on table and indexes, and well as individual columns and partitions of tables. It does not gather cluster statistics; however, you can use DBMS_STATS to gather statistics on the individual tables instead of the whole cluster.

You can use the DBMS_STATS package or the ANALYZE statement to gather statistics about the physical storage characteristics of a table, index, or cluster. These statistics are stored in the data dictionary and can be used by the optimizer to choose the most efficient execution plan for SQL statements accessing analyzed objects.

When you generate statistics for a table, column, or index, if the data dictionary already contains statistics for the object, then Oracle updates the existing statistics. The older statistics are saved and can be restored later if necessary.

TEST ENVIRONMENT

Database: Oracle 10g Enterprise Edition 10.2.0.1.0
Server: Ubuntu 7.04 Desktop, on VMware 5.5.3, 512 MB of RAM, Pentium M 1.7 Ghz
Client: Oracle 10g XE Client, SQL*Plus on Windows Console

TEST CODE EXECUTION AND RESULTS

SQL> conn test/test@bilalEE
Connected.
SQL> set autotrace traceonly
SQL> set timing on
SQL> alter session set statistics_level=BASIC;

Session altered.

Elapsed: 00:00:00.08
SQL> drop index t1_inx_owner;
drop index t1_inx_owner
           *
ERROR at line 1:
ORA-01418: specified index does not exist


Elapsed: 00:00:00.03
SQL> drop table t1 purge;
drop table t1 purge
           *
ERROR at line 1:
ORA-00942: table or view does not exist


Elapsed: 00:00:01.91
SQL> create index t1_inx_owner on t1(owner);

Index created.

Elapsed: 00:00:05.79
SQL> select table_name,num_rows,blocks,empty_blocks,last_analyzed from all_tables
where table_name='T1';

TABLE_NAME                       NUM_ROWS     BLOCKS EMPTY_BLOCKS LAST_ANAL
------------------------------ ---------- ---------- ------------ ---------
T1

Elapsed: 00:00:01.79
SQL> select index_name, table_name, blevel, leaf_blocks, clustering_factor, num_rows,
 last_analyzed from all_indexes where index_name='T1_INX_OWNER';

INDEX_NAME                     TABLE_NAME                         BLEVEL LEAF_BLOCKS
------------------------------ ------------------------------ ---------- -----------
CLUSTERING_FACTOR   NUM_ROWS LAST_ANAL
----------------- ---------- ---------
T1_INX_OWNER                   T1                                      2         687
             6214     292428 23-JUL-07

Elapsed: 00:00:01.10
SQL> alter system flush buffer_cache;

System altered.

Elapsed: 00:00:00.03
SQL> select name from t1 where owner='SYSMAN';

128423 rows selected.

Elapsed: 00:00:03.63

Execution Plan
----------------------------------------------------------
Plan hash value: 3617692013

--------------------------------------------------------------------------
| Id  | Operation         | Name | Rows  | Bytes | Cost (%CPU)| Time     |
--------------------------------------------------------------------------
|   0 | SELECT STATEMENT  |      |   181K|  6028K|  1391   (2)| 00:00:17 |
|*  1 |  TABLE ACCESS FULL| T1   |   181K|  6028K|  1391   (2)| 00:00:17 |
--------------------------------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------

   1 - filter("OWNER"='SYSMAN')

Note
-----
   - dynamic sampling used for this statement


Statistics
----------------------------------------------------------
        244  recursive calls
          0  db block gets
       8835  consistent gets
       6197  physical reads
          0  redo size
     871122  bytes sent via SQL*Net to client
      28621  bytes received via SQL*Net from client
       2570  SQL*Net roundtrips to/from client
          5  sorts (memory)
          0  sorts (disk)
     128423  rows processed

SQL> alter system flush buffer_cache;

System altered.

Elapsed: 00:00:00.05
SQL> select name from t1 where owner='SYSMAN';

128423 rows selected.

Elapsed: 00:00:02.81

Execution Plan
----------------------------------------------------------
Plan hash value: 3617692013

--------------------------------------------------------------------------
| Id  | Operation         | Name | Rows  | Bytes | Cost (%CPU)| Time     |
--------------------------------------------------------------------------
|   0 | SELECT STATEMENT  |      |   181K|  6028K|  1391   (2)| 00:00:17 |
|*  1 |  TABLE ACCESS FULL| T1   |   181K|  6028K|  1391   (2)| 00:00:17 |
--------------------------------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------

   1 - filter("OWNER"='SYSMAN')

Note
-----
   - dynamic sampling used for this statement


Statistics
----------------------------------------------------------
          0  recursive calls
          0  db block gets
       8729  consistent gets
       6174  physical reads
          0  redo size
     871122  bytes sent via SQL*Net to client
      28621  bytes received via SQL*Net from client
       2570  SQL*Net roundtrips to/from client
          0  sorts (memory)
          0  sorts (disk)
     128423  rows processed

SQL> analyze table t1 compute statistics;

Table analyzed.

Elapsed: 00:00:07.27

SQL> select table_name,num_rows,blocks,empty_blocks,last_analyzed,sample_size
from all_tables where table_name='T1';

TABLE_NAME                       NUM_ROWS     BLOCKS EMPTY_BLOCKS LAST_ANAL SAMPLE_SIZE
------------------------------ ---------- ---------- ------------ --------- -----------
T1                                 292432       6281          119 27-JUL-07      292432

Elapsed: 00:00:00.27
SQL> alter system flush buffer_cache;

System altered.

Elapsed: 00:00:00.46
SQL> select name from t1 where owner='SYSMAN';

128423 rows selected.

Elapsed: 00:00:01.22

Execution Plan
----------------------------------------------------------
Plan hash value: 3177033591

--------------------------------------------------------------------------------------
| Id  | Operation                   | Name         | Rows  | Bytes | Cost (%CPU)| Time
--------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT            |              | 15391 |   270K|   368   (1)| 00:0
|   1 |  TABLE ACCESS BY INDEX ROWID| T1           | 15391 |   270K|   368   (1)| 00:0
|*  2 |   INDEX RANGE SCAN          | T1_INX_OWNER | 15391 |       |    39   (0)| 00:0
--------------------------------------------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------

   2 - access("OWNER"='SYSMAN')


Statistics
----------------------------------------------------------
          1  recursive calls
          0  db block gets
       6878  consistent gets
       1762  physical reads
          0  redo size
     871122  bytes sent via SQL*Net to client
      28621  bytes received via SQL*Net from client
       2570  SQL*Net roundtrips to/from client
          0  sorts (memory)
          0  sorts (disk)
     128423  rows processed

SQL> alter system flush buffer_cache;

System altered.

Elapsed: 00:00:00.02
SQL> select name from t1 where owner='SYSMAN';

128423 rows selected.

Elapsed: 00:00:01.05

Execution Plan
----------------------------------------------------------
Plan hash value: 3177033591

--------------------------------------------------------------------------------------
| Id  | Operation                   | Name         | Rows  | Bytes | Cost (%CPU)| Time
--------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT            |              | 15391 |   270K|   368   (1)| 00:0
|   1 |  TABLE ACCESS BY INDEX ROWID| T1           | 15391 |   270K|   368   (1)| 00:0
|*  2 |   INDEX RANGE SCAN          | T1_INX_OWNER | 15391 |       |    39   (0)| 00:0
--------------------------------------------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------

   2 - access("OWNER"='SYSMAN')


Statistics
----------------------------------------------------------
          0  recursive calls
          0  db block gets
       6878  consistent gets
       1762  physical reads
          0  redo size
     871122  bytes sent via SQL*Net to client
      28621  bytes received via SQL*Net from client
       2570  SQL*Net roundtrips to/from client
          0  sorts (memory)
          0  sorts (disk)
     128423  rows processed

Download Code

You can download the presentation and presentation notes from here.

COMPARISON

CONCLUSION

Here, we see in the first case, Oracle didn't have any statistics and idea about the data, so, to use the CBO, it uses dynamic sampling (randomly reads some data from table and makes a decision from that statistics, before executing the query) and decides to full table scan. The strange thing is, although Oracle has statistics for the index on the table, and they are true, real an up-to-date statistics, it did not use them. When we gather table statistics, CBO uses it and then decided to use index.

• For CBO, having the most up-to-date statistics is important to have a better decision

• Don’t forget the extra cost of gathering statistics (especially for large tables)

• Oracle automatically gathers *some* statistics for its internal use and for CBO, that you can’t turn off (10g). We can turn off some statistics, also we can gather extra statistics (histograms)

Feed users, click here to go to this topic's page. You can find related topics and reference links and you can read the comments for this topic

Feed users, click here to post a comment for this topic

Note: There might be problems about HTML code and parsing and displaying HTML code of your viewer application. Also, there are problems caused by Google Reader on displaying of code samples on the entry. Also, the entry might be updated. So, it is recommended to visit the page of the entry.

Turkcell Staj Günlüğü - 9: "SQL, PL/SQL and Java" ve "Redo Internals"

Bilal Hatipoglu — 30.07.2007 09:52:30

Merhaba, yazı dizine stajın 3. haftasından devam ediyoruz. 18.07.2007 günü Alp Emre'nin "SQL, PL/SQL and Java" konulu sunumu vardı. Basit bir konu aslında, temel olarak SQL'den, PL/SQL'den ve Java'dan bahsedildi. Aslında her biri başlı başına bir eğitim konusu fakat bu bir introduction tadındaydı :)

Bu konuda bahsedilecek çok fazla birşey yok, sunum da gayet güzel hazırlanmış zaten. Alp Emre'nin sunumunu buradan indirebilirsiniz. Bende konu hakkında bir kaç notumu paylaşacağım:

• SQL Database'lerin ortak dilidir. Database'ler SQL'den anlar. Makine dili Assembly'dir. Daha yüksek seviye programlama dillerinde yazdığımız bütün kodlar assembly'e çevrilir, ve makinede bu şekilde çalıştırılır. Bunun gibi, her ne kadar bazı uygulamalar daha yüksek seviye kontroller sunsa da, temelde SQL aracılığıyla database'e erişirler.

• Bir çok farklı database olduğu için SQL'in de bir standardı geliştirilmiştir. Bu standardın adı Ansi SQL'dir. Oracle, Ansi SQL'i tamamen desteklemekle birlikte, bunu kendi bir çok komutuyla da zenginleştirmiştir. Oracle'a has komutları içermeyen SQL'ler, Ansi SQL'i destekleyen diğer bir çok database uygulamasında da çalışacaktır.

• SQL'i işlevine göre bir çok kategoriye ayırabiliriz. Bu kategoriler ve tanımlarına girmeyeceğim. Sunumda hepsi var zaten.

• Sunumda, cursor'lardan ve shared SQL'den de bahsedildi. Daha öncki sunumları, özellikle Ersin'in sunumunu takip edenler cursorlara yabancı değillerdir. Ersin'in "Statement Processing" sunumuyla ilgili günlüğe buradan, cursorlar hakkında geniş bilgi içeren bir başka makaleme ise buradan ulaşabilirsiniz.

• SQL'in nasıl parse edildiği ve çalıştırılırken hangi adımlardan geçtiği yine Ersin'in sunumu'nda daha ayrıntılı olarak anlatıldı. Okumayanların o sunuma da göz atmasını tavsiye ederim.

• PL/SQL, Oracle'ın prosedürel dilidir. SQL'in bir programlama dili gibi geliştirilmiş halidir. PL/SQL'de normal programlama dilleri gibi döngüler, fonksiyonlar, prosedürler, paketler ve değişkenler mevcuttur. Fakat yine de temelde database ile bağlantı kurulan kısımlar PL/SQL blokları içinde de olsa SQL cümleleridir. Database'in dilinin SQL olduğunu daha önce belirtmiştim. Developer'ların işi de zaten büyük oranda bu PL/SQL kodlarını, paketleri ve prosedürleri yazmaktır.

• PL/SQL paketleri, bir çok prosedür'ü içinde bulunduran bir yapıdır. Java'daki Package'lara benzetebiliriz. Bu kod ve paketler, hem text olarak hem de compile edilmiş vaziyette database'de saklanır. Bir paket ilk çalıştırıldığı an, bir kez compile edilir. İçinde hata bulunan paketler invalid duruma düşerek her çağrıldığında compile edilmeye çalışılır.

• Java'dan da biraz bahsedildi. Aslında bu sunumdaki Java kısmı bir Java dersi şeklinde değildi. Zaten Java'nın hiçbir kısmını 1 saatlik bir sunumda anlatmak mümkün değil :) Sunumda anlatılan, Oracle'ın Java Stored Procedure'ları desteklediğidir. Oracle üzerinde Java kodları saklanarak çalıştırılabilir. Bu Java kodlu prosedürlere Java Stored Procedures denir. Oracle'ın kendi içinde bir JVM'i vardır ve Java kodlarını bu şekilde çalıştırır. Burada önemli nokta ise, Java her ne akdar desteklense de Oracle'a uzak bir dil olduğundan ve JVM üzerinden çalıştığından PL/SQL'e göre daha yavaş çalışır. Bu yüzden PL/SQL paketleri ile yapabileceğimiz herşeyi, özel olarak ihtiyaç duymuyorsak PL/SQL ile yapmalıyız.

Alp Emre'nin sunumu bunları içeriyordu. Sunuma bir göz atmanızı öneririm. Alp Emre'nin sunumunu buradan indirebilirsiniz. Aynı zamanda geçen senelere ait, Seven'in SQL konulu sunumu ve Efficient SQL konulu sunumlarını, TonguÇ'un "PL/SQL ve Java" konulu sunumunu, Hüsnü'nün PL/SQL konulu sunumunu, Burak ÇELEBİ'nin "PL/SQL Best Practices" konulu sunumunu da buradan indirebilirsiniz. Bu sunumlarda çok daha fazla bilgiye ulaşabilirsiniz.

19.07.2007 perşembe günü oldukça yoğun şanssızlıkların ardından Ersin'in gerçekten çok iyi hazırlanmış olduğuna emin olduğum workshop'ının ertelenmesiyle sonuçlandı.

Haftanın son gününde Mert İNAN'ın sunumu vardı. Konu, Mert'in ilk haftalarda yaptığı Undo/Redo ve Transaction management konusunun advanced haliydi: "Redo Internals". Mert, tabi haliyle bu konuda bir kitap falan yazmamış olduğu için, Julian DYKE'ın sunumu üzerinden gitti.

Yanlız baştan uyarayım, sunum fazlasıya advanced bir sunum, dolayısıyla anlamadığınız yerlere takılmayın, anladığınız yerlere, animasyon falan olan neşeli yerlere bir göz gezdirin sadece :)

Şu an sunumu tekrar inceleyerek gözüme takılan, aklıma gelen ufak tefek notları sizinle paylaşacağım:

• "Redo log nedir?" sorusunun cevabıyla başlamak gerekiyor sanırım. Redo log'lar, database'deki bütün değişiklikleri tutan yapılardır. Peki redo log'lar ne işe yarar? En önemli görevleri sanırım database'in anormal sonlanması(elektrik kesilmesi vs.) durumunda instance recovery için kullanılmalarıdır. Bunun dışında backup için de kullanılırlar. Şöyle ki: Database'inizin dün alınan bir yedeği varsa ve bugün de yedeğini almak istiyorsanız, bütün database'in yedeğini almak yerine sadece bugüne ait redo dosyalarını alıp dünkü yedeğinize uygulayarak database'inizin bugünkü haline ulaşabilirsiniz. Ayrıca bir de, flashback veya normal recovery esnasında undo log'larla birlikte kullanılırlar. Redo log'lar database'imiz için kritiktir.

• Redo loglar, database üzerinde yapılan her değişikliği kaydeder. Aslında her değişikliğin son hali redo log'larda, önceki hali undo log'larda tutulur. Redo log'lara niye ihtiyacımız olduğu konusuna biraz daha değinelim:

Redo log'lar aslında memory'mizin yedeği gibidir. Oracle hiçbir zaman o anda yapması gerekmeyen, maliyetli bir işi o anda yapmaz. Datadaki her değişikliğin diske yazılmadan önce memory'de tutulduğunu biliyoruz. Yoksa her seferinde diske gitseydi zaten bu çok maliyetli olurdu. Undo log'larda aynı şekilde dike yazılmadan önce memory'de tutulur. Temelde problem, anormal bir sonlanma (elektrik kesilmesi, diskin uçması, OS'nin uçması, makinenin reset yemesi, Oracle processlerinin zorla sonlandırılması vs.) durumunda memory'nin uçmasıdır. Bu durumda Oracle'ı tekrar başlattığımızda bellekteki verileri orada bulamayacağız. İşte bu yüzden, bellekteki verilerin hepsine değil de, bizim için kritik olanlara, bizim için kritik olan zamanlarda ihityacımız var. Örneğin, commit edilen veri, henüz datafile'lara yazılmamışken makine uçsa bile, databse'in tutarlılığı için onlara ihtiyacımız var. Redo log'lar işte bu memory'deki verilerin yedeğidir. Oracle mecbur kalmadığı sürece belli aralıklarda ve mecbur kaldığı an derhal bu redo log'ları diske yazar. Redo log'lar anormal sonlanma sonrasında Oracle yeniden açıldığında, memory'deki kritik bilgileri tekrar memory'e yükleyerek (adeta sonlanmadan önceki memory verisini geri getirerek) hayatımızı kurtarır :)

• Redo log'lar data'daki her değişiklik için üretilir. DML komutları, DDL komutları ve recursive statementlar, hepsi redo log üretir.

• Redo log'ları diske Oracle'ın LGWR(Log Writer) isimli bir prosesi yazar.

• Online redo log'lar ve Archive redo log'lar var. Anlık(online) Redo log'lar normalde diskte bir kaç dosya üzerindedir. Bu dosyalar sırayla kullanılır. Sonuncu da yazılıp dolduğunda yazma işlemine baştakinden devam edilir. Bu ekilde log'lar diske yazılır. Fakat bu durumda, bütün log dosyaları dolup baştakine yazmaya başladığımızda, aslında daha önceki redo verisini kaybetmeye başlarız. Geçmişteki bu redo verisi bizim için kritikse, onun üzerine yazılmadan önce o arşivlenmeli. İşte Archive redo log'lar da, bu Online redo log'ların arşivlenmiş kopyalarıdır. ARşiv redo log'larını yazmaktan ise ARCH(Archiver) isimli Oracle prosesi sorumludur. Sunumun6. sayfasında, bu redo log'ların yazılması ver arşivlenmesini çok güzel anlatan animasyonlu bir slayt var.

• Sunumda daha sonra, Redo log'ların iç yapısına girilmiş. hangi operasyondan sonra nasıl bir redo verisi üretilir, onun üzerinde durularak tek tek örneklenmiş. Bir developer için (hatta normal bir DBA için bile) bunlar çok fazla ayrıntı. O yüzden bunlara tek tek girmeyeceğim. Fakat siz de benim gibi ille mantığını en ince ayrıntısına kadar hemen öğrenme manyağı(güzel tanım oldu bu ya aslında :) ) iseniz sunumu incelemeye devam edebilirsiniz. Arada sıkılırsanız kendinize mola verip sonr devam edin, çünkü sıkılacaksınız :)) Sunumun ilerleyen kısımlarında, dediğim, hangi operasyondan ne tip bir redo verisi üretilir konusu ilginç. Ama "banane ulan" diyorsanız, Oracle'ın, her operasyon için, o operasyonu tekrar gerçekleştirebilecek ve verinizi kurtaracak kadar (ama kesinlikle daha fazlasını değil) bilgiyi redo log'larda sakladığını bilmeniz yeterli.

Julian DYKE'ın sunumu da bu şekilde sürüp gidiyor. Sunumu bu linkten indirebilirsiniz. Fakat bundan önce, eğer bakmadıysanız Mert'in transaction'ın ne olduğu konusundaki sunumuna ait makalemi de incelemelisiniz. Bir de, bu redo konusunu daha basit ve daha iyi anlamak için, KYTE'ın "Expert Database Architecture" kitabının 9. Chapter'ını incelemenizi kesinlike öneririm. DYKE'ın sunumu sizi baydıysa bile, siz yine de KYTE'ın kitabına kesinlikle göz atın :))

Konular ilerledikçe, birbiriyle bağlantılı konuların sayısı artıyor ve konular birbirini tamamlıyor hale geliyor dikkat ederseniz. Zaten bir hafta sonraki konulardan sonra temelde "Concepts Guide" ve KYTE'ın kitaplarındaki önemli chapter'lar bitmiş olacak.

Herkese iyi çalışmalar.

Feed users, click here to go to this topic's page. You can find related topics and reference links and you can read the comments for this topic

Feed users, click here to post a comment for this topic

Note: There might be problems about HTML code and parsing and displaying HTML code of your viewer application. Also, there are problems caused by Google Reader on displaying of code samples on the entry. Also, the entry might be updated. So, it is recommended to visit the page of the entry.

Implicit vs. Explicit cursors - Performance analysis

Bilal Hatipoglu — 24.07.2007 21:23:24

For every SQL statement execution certain area in memory is allocated on the Shared Pool portion of the SGA. PL/SQL allow you to name this area. This private SQL area is called context area or cursor. A cursor acts as a handle or pointer into the context area. A PL/SQL program controls the context area using the cursor. Cursor represents a structure in memory and is different from cursor variable.

When you declare a cursor, you get a pointer variable, which does not point any thing. When the cursor is opened, memory is allocated and the cursor structure is created. The cursor variable now points the cursor. When the cursor is closed the memory allocated for the cursor is released.

Cursors allow the programmer to retrieve data from a table and perform actions on that data one row at a time. There are two types of cursors implicit cursors and explicit cursors.

IMPLICIT CURSORS

PL/SQL issues an implicit cursor whenever you execute a SQL statement directly in your code, as long as that code does not employ an explicit cursor. It is called an "implicit" cursor because you, the developer, do not explicitly declare a cursor for the SQL statement.

In PL/SQL, you can refer to the most recent implicit cursor as the SQL cursor, which always has the attributes %FOUND, %ISOPEN, %NOTFOUND, and %ROWCOUNT.

EXPLICIT CURSORS

An explicit cursor is a SELECT statement that is explicitly defined in the declaration section of your code and, in the process, assigned a name. There is no such thing as an explicit cursor for UPDATE, DELETE, and INSERT statements.

With explicit cursors, you have complete control over how to access information in the database. You decide when to OPEN the cursor, when to FETCH records from the cursor (and therefore from the table or tables in the SELECT statement of the cursor) how many records to fetch, and when to CLOSE the cursor. Information about the current state of your cursor is available through examination of the cursor attributes. This granularity of control makes the explicit cursor an invaluable tool for your development effort.

An explicit cursor is a named query that is defined as a cursor in PL/SQL by use of the keyword CURSOR. UPDATE, DELETE and INSERT statements always use an implicit cursor (because they are not named).

The difference between explicit and implicit cursors in Oracle PL SQL is that explicit cursors give you complete control over opening, closing and fetching from an explicit cursor, including the number of rows fetched. Also, fetching from an explicit cursor will never raise a NO_DATA_FOUND or a TOO_MANY_ROWS exception.

TEST ENVIROMENT

Database: Oracle 10g Express edition 10.2.0.1.0
Server: Windows XP Proffessional, 512 MB of RAM, P4 2.4 Ghz
Client: Same machine, SQL*Plus on Windows Console

TEST CASE #1

create or replace procedure impl_cursor as
     v_row dba_source%ROWTYPE;
begin
     select * into v_row from dba_source where line=9999;
     dbms_output.put_line(v_row.text);
end;

create or replace procedure expl_cursor as
       v_row dba_source%ROWTYPE;
       cursor test_cur is select * from dba_source where line=9999;
begin
     open test_cur;
     fetch test_cur into v_row;
     close test_cur;
     dbms_output.put_line(v_row.text);
end;

SQL> exec runStats_pkg.rs_start;

PL/SQL procedure successfully completed.

SQL> exec expl_cursor;

PL/SQL procedure successfully completed.

SQL> exec runStats_pkg.rs_middle;

PL/SQL procedure successfully completed.

SQL> exec impl_cursor;

PL/SQL procedure successfully completed.

SQL> exec runStats_pkg.rs_stop;
Run1 ran in 14 hsecs
Run2 ran in 10 hsecs
run 1 ran in 140% of the time

Name                                  Run1        Run2        Diff
STAT...free buffer requested             0           1           1
STAT...opened cursors cumulati           4           5           1
STAT...execute count                     6           7           1
LATCH.undo global data                   5           6           1
LATCH.cache buffers lru chain            1           0          -1
LATCH.KMG MMAN ready and start           0           1           1
STAT...session cursor cache hi           4           5           1
STAT...table scans (short tabl           1           2           1
STAT...parse count (total)               4           5           1
STAT...bytes received via SQL*       1,141       1,139          -2
LATCH.messages                           0           2           2
STAT...active txn count during           4           6           2
STAT...cleanout - number of kt           4           6           2
STAT...calls to kcmgcs                   4           6           2
LATCH.object queue header oper           2           0          -2
STAT...redo entries                      9          11           2
STAT...CPU used by this sessio          10           7          -3
LATCH.library cache                     61          65           4
LATCH.library cache pin                 48          52           4
STAT...recursive cpu usage              10           6          -4
STAT...Elapsed Time                     16          12          -4
STAT...calls to get snapshot s           6          11           5
STAT...consistent changes               17          23           6
STAT...recursive calls                   4          10           6
STAT...db block gets                    17          25           8
STAT...db block gets from cach          17          25           8
STAT...db block changes                 26          35           9
STAT...bytes sent via SQL*Net          650         637         -13
STAT...CPU used when call star          23           7         -16
STAT...DB time                          27           7         -20
LATCH.simulator lru latch              189         236          47
LATCH.simulator hash latch             189         236          47
STAT...undo change vector size       2,128       2,268         140
STAT...index fetch by key            4,086       4,244         158
STAT...buffer is pinned count        8,124       8,439         315
STAT...consistent gets - exami       8,177       8,495         318
STAT...redo size                     2,776       3,120         344
STAT...table scan blocks gotte         596       1,222         626
STAT...no work - consistent re         598       1,224         626
STAT...consistent gets               8,784       9,733         949
STAT...consistent gets from ca       8,784       9,733         949
STAT...session logical reads         8,801       9,758         957
LATCH.cache buffers chains           9,489      11,074       1,585
STAT...table scan rows gotten       50,650     103,488      52,838

Run1 latches total versus runs -- difference and pct
Run1        Run2        Diff       Pct
10,226      11,914       1,688     85.83%

PL/SQL procedure successfully completed.

Download Code

TEST CASE #2

create or replace procedure impl_cursor_bulk as
     TYPE test1_tab IS TABLE OF dba_source%ROWTYPE;
     t_tab  test1_tab;
begin
     select * bulk collect into t_tab from dba_source;
     dbms_output.put_line(t_tab.COUNT);
end;

create or replace procedure expl_cursor_bulk as
     TYPE test1_tab IS TABLE OF dba_source%ROWTYPE;
     t_tab  test1_tab;
       cursor test_cur is select * from dba_source;
begin
     open test_cur;
     fetch test_cur bulk collect into t_tab;
     close test_cur;
     dbms_output.put_line(t_tab.COUNT);
end;

SQL> exec runStats_pkg.rs_start;

PL/SQL procedure successfully completed.

SQL> exec expl_cursor_bulk;
292566

PL/SQL procedure successfully completed.

SQL> exec runStats_pkg.rs_middle;

PL/SQL procedure successfully completed.

SQL> exec impl_cursor_bulk;
292566

PL/SQL procedure successfully completed.

SQL> exec runStats_pkg.rs_stop;
Run1 ran in 267 hsecs
Run2 ran in 213 hsecs
run 1 ran in 125,35% of the time

Name                                  Run1        Run2        Diff
LATCH.parameter table allocati           0           1           1
LATCH.resmgr:free threads list           0           1           1
LATCH.resmgr:actses active lis           0           1           1
LATCH.process group creation             0           1           1
LATCH.process allocation                 0           1           1
LATCH.simulator hash latch           1,700       1,701           1
STAT...physical read total IO          334         335           1
STAT...recursive calls                   3           2          -1
LATCH.simulator lru latch              731         732           1
LATCH.Consistent RBA                     0           1           1
LATCH.lgwr LWN SCN                       0           1           1
LATCH.mostly latch-free SCN              0           1           1
LATCH.FOB s.o list latch                 0           1           1
LATCH.OS process: request allo           0           1           1
LATCH.OS process allocation              1           2           1
STAT...physical read IO reques         334         335           1
LATCH.channel handle pool latc           0           1           1
LATCH.session timer                      1           2           1
LATCH.dummy allocation                   0           1           1
STAT...physical reads cache pr       4,794       4,796           2
STAT...consistent gets - exami           7           9           2
STAT...physical read total mul         331         333           2
STAT...calls to kcmgcs                   4           6           2
STAT...cleanout - number of kt           4           6           2
STAT...active txn count during           4           6           2
STAT...bytes received via SQL*       1,146       1,144          -2
LATCH.JS slv state obj latch             0           2           2
LATCH.redo writing                       3           6           3
LATCH.cache buffers lru chain        5,128       5,131           3
STAT...physical reads                5,128       5,131           3
LATCH.OS process                         0           3           3
STAT...physical reads cache          5,128       5,131           3
STAT...redo entries                      9          12           3
STAT...free buffer requested         5,128       5,131           3
LATCH.compile environment latc           3           0          -3
LATCH.In memory undo latch               6           2          -4
STAT...consistent gets from ca       6,419       6,423           4
LATCH.session allocation                 6           2          -4
STAT...consistent gets               6,419       6,423           4
LATCH.multiblock read objects          662         666           4
LATCH.object queue header oper      10,257      10,263           6
LATCH.library cache lock                12           6          -6
LATCH.messages                          15          21           6
STAT...workarea memory allocat          -3           3           6
LATCH.shared pool                        5          11           6
STAT...db block gets from cach          17          24           7
STAT...consistent changes               17          24           7
STAT...db block gets                    17          24           7
STAT...free buffer inspected         5,129       5,121          -8
STAT...hot buffers moved to he           9           0          -9
STAT...db block changes                 26          36          10
LATCH.active service list                0          11          11
STAT...session logical reads         6,436       6,447          11
STAT...bytes sent via SQL*Net          644         631         -13
LATCH.session idle bit                  41          27         -14
LATCH.dml lock allocation               16           0         -16
STAT...CPU used by this sessio         233         209         -24
STAT...recursive cpu usage             224         200         -24
LATCH.cache buffers chains          18,101      18,075         -26
STAT...CPU used when call star         244         209         -35
LATCH.JS queue state obj latch           0          36          36
STAT...user I/O wait time               48           8         -40
LATCH.library cache pin                104          54         -50
STAT...Elapsed Time                    269         214         -55
LATCH.row cache objects                135          78         -57
LATCH.enqueue hash chains               18          79          61
LATCH.library cache                    139          69         -70
STAT...DB time                         282         210         -72
LATCH.enqueues                           5          79          74
STAT...undo change vector size       2,136       2,340         204
STAT...redo size                     2,784       3,204         420
STAT...physical read bytes      42,008,576  42,033,152      24,576
STAT...physical read total byt  42,008,576  42,033,152      24,576

Run1 latches total versus runs -- difference and pct
Run1        Run2        Diff       Pct
37,491      37,472         -19    100.05%

PL/SQL procedure successfully completed.

Download Code

TEST CASE #3

create table tcur ( x number primary key ) ;

insert into tcur
select rownum x 
from all_objects
where rownum < 1000;

analyze table tcur compute statistics;

create or replace procedure imp_test is
 y number;
begin
  for i in 1 .. 50 loop
    for j in 1 .. 999 loop
        select x
        into   y
        from   tcur 
        where  x = j;
    end loop;
  end loop;
end;
/

create or replace procedure exp_test is
  cursor c(p number) is
    select x 
    from tcur
    where x = p;
 y number;
begin
  for i in 1 .. 50 loop
    for j in 1 .. 999 loop
       open c(j);
       fetch c into y;
       close c;
    end loop;
  end loop;
end;
/

SQL> set timing on
SQL> exec imp_test

PL/SQL procedure successfully completed.

Elapsed: 00:00:02.15

SQL> alter system flush buffer_cache;

System altered.

SQL> alter system flush shared_pool;

System altered.

SQL> exec exp_test

PL/SQL procedure successfully completed.

Elapsed: 00:00:03.26

Download Code

You can download the presentation and presentation notes from here.

You can also download the 10046 SQL Trace and tkprof output files of the test case #3 from here.

CONCLUSION

Pick any tuning manual, and you'll often see a section about coding with explicit cursors as opposed to implicit ones. The argument seems sound, typically going along the lines of:

"With an explicit cursor, you can open, fetch and close, whereas with an implicit cursor must issue open, fetch, second fetch, close - the second fetch being needed to check for a too many rows exception.

However, since 7.3, those cunning people at Oracle have worked around this problem and the second fetch is no longer required. Because PL/SQL is interpreted (watch out for the new native compile option in 9i), implicit cursors will actually run FASTER than explicit since typically you are using less code to achieve the same task.

Performance: Nearly same.

If you use an explicit cursor, you are more likely (or at least able) to reuse that cursor, which increases the chance that it will be pre-parsed in shared memory when needed.

In previous versions, implicit cursors cannot handle multiple rows. You have to be sure that a particular SELECT statement will return only one row: That’s an old issue!

Implicit cursors are easier to handle and less code to write.

Explicit cursors give you complete control over opening, closing and fetching from an explicit cursor, including the number of rows fetched.

If you don’t have to use explicit cursors, use implicit!

Feed users, click here to go to this topic's page. You can find related topics and reference links and you can read the comments for this topic

Feed users, click here to post a comment for this topic

Note: There might be problems about HTML code and parsing and displaying HTML code of your viewer application. Also, there are problems caused by Google Reader on displaying of code samples on the entry. Also, the entry might be updated. So, it is recommended to visit the page of the entry.

When a transaction begins?

Bilal Hatipoglu — 24.07.2007 19:02:39

A transaction is a logical unit of work that contains one or more SQL statements. A transaction is an atomic unit. The effects of all the SQL statements in a transaction can be either all committed (applied to the database) or all rolled back (undone from the database).

Transaction management is the main mechanism that distinguishes between DBMS from other DBMS and DBMS from file system. The main purpose is to take the database from one consistent state to next. This satisfies the consistency of database for many cuncurrent users. That is imposible with file systems.

In Oracle, every manipulation statement must be in a transaction, so whether you give a "set transaction" command or not, Oracle opens a transaction. It is obvious that a transaction ends with a commit or rollback command. But it actually is not obvious that when a transaction begins.

Many peope say that "A transaction begins with first DML statement" or like that. I will show you some examples and some experimental about that. But first, we will look at some descriptions about that.

In Oracle 10gR2 Documentation, it is written like "A transaction begins with the first executable SQL statement". That is ambiguous, what is an "executable statement". In what conditions a transaction begins.

Also some people writes like "A new transaction begins when a session is opened and after each rollback or commit". That is really wrong, no doubt about it :)

And I see some descriptions about this topic like "Transaction begins at execution of first data change" or like "A transaction begins with the first statement that gets a TX lock" (that is first statement that modifies data). These are actually true. I will illustrate with examples now. There are a lot of things to understand for knowing people. I will not comment for each of them.

SQL> conn test@bilalEE as sysdba
Connected.
SQL> select * from v$transaction;

no rows selected

SQL> create table t as select rownum n from all_objects where rownum < 1000;

Table created.

SQL> select * from v$transaction;

no rows selected

SQL> select * from t where n=1;

         N                                                                      
----------                                                                      
         1                                                                      

SQL> select * from v$transaction;

no rows selected

SQL> update t set n=2000 where n=3000;

0 rows updated.

SQL> select * from v$transaction;

no rows selected

SQL> delete from t where n=3000;

0 rows deleted.

SQL> select * from v$transaction;

no rows selected

SQL> alter table t add constraint pk_t primary key(n);

Table altered.

SQL> select * from v$transaction;

no rows selected

SQL> insert into t values(500);
insert into t values(500)
*
ERROR at line 1:
ORA-00001: unique constraint (SYS.PK_T) violated 


SQL> select * from v$transaction;

no rows selected

SQL> create trigger trig_t before insert on t
  2  begin
  3  raise_application_error(-20001,'trigger not allows to insert to this table');
  4  end;
  5  /

Trigger created.

SQL> select * from v$transaction;

no rows selected

SQL> insert into t values(1001);
insert into t values(1001)
            *
ERROR at line 1:
ORA-20001: trigger not allows to insert to this table
ORA-06512: at "TEST.TRIG_T", line 2
ORA-04088: error during execution of trigger 'TEST.TRIG_T'


SQL> select * from v$transaction;

no rows selected

SQL> savepoint sp1;

Savepoint created.

SQL> select * from v$transaction;

no rows selected

SQL> create procedure test_p as
  2  begin
  3  dbms_output.put_line('selam');
  4  end;
  5  /

Procedure created.

SQL> select * from v$transaction;

no rows selected

SQL> exec test_p;
selam

PL/SQL procedure successfully completed.

SQL> select * from v$transaction;

no rows selected

SQL> update t set n=1000 where n=999;

1 row updated.

SQL> select * from v$transaction;

ADDR         XIDUSN    XIDSLOT     XIDSQN     UBAFIL     UBABLK     UBASQN     UBAREC ST
-------- ---------- ---------- ---------- ---------- ---------- ---------- ---------- --
288E7124          8         44        305          2       1309        203         58 AC

SQL>

Download Code

Turkcell Staj Günlüğü - 8: Statement Processing and CBO

Bilal Hatipoglu — 24.07.2007 11:44:03

Merhaba, yazı dizinde stajımın 3. haftasına geçtik. (gerçekte şu anda 4. haftadayız) 16.07.2007 haftasına aslında Ersin damgasını vurdu diyebiliriz :)) Peş peşe pazartesi ve salı günü "Statement Processing and CBO" sunumları vardı. Gerçekten güzel sunumlardı, Ersin'in sunumlarına bayılıyorum zaten, anlatımı hoş oluyor :)

Bu makalede de Ersin'in sunumlarından bahsedeceğim, yani konumuz "Statement processing and CBO". Bu da önemli bir konu diyeceğim ama diyeceksiniz ki "bütün konulara önemli diyorsun sende önemsiz konu yokmu"... Bende cevap veriyorum ki "önemsiz olsa hiç Tonguç eğitim programına koyar mıydı hiç" :)) Dolayısıyla artık bu konular önemli, neden önemli, çünkü Oracle'ı Oracle yapan şeyler bunlar geyiğine girmeyeceğim. Zaten CBO'nun ne olduğunu anlayınca Oracle'ın neden bu kadar değerli olduğunu bir kez daha anlıyor insan. CBO'nun ne olduğunu anlamak derken, nasıl çalıştığını anlamak imkansız zaten :) Jonathan Lewis denen bir amca var, o kafayı yoruyor bu konulara senelerdir, ama o da hala yaza yaza bitiremediğine göre ve hala aklı başında olduğuna göre biz zaten sadece tanımını anlamakla yetiniriz. Ha, ben biraz daha ileri gidicem diyorsanız alın bir 10053 Trace, o zaman zaten "agugu" yapıp dudağınızla oynamaya ve kafanıza bir huni takıp etrafta dolaşmaya başlarsınız :))

Neyse gelelim konumuza biz, Ersin böylesine zor bir konuyu gerçekten güzel anlattı. Ha, birde perşembe günü bir workshop yapacaktı ama bilinmeyen nedenlerden dolayı :P nedense artık, sunumu iptal etmek zorunda kaldık :P Şaka bir yana, Ersin buna çok iyi hazırlanmıştı fakat gün kötü bir gündü, herşey ters gitti ve en sonunda Ersin örnekleri çalıştıracağı sunucuya da bağlanamayınca "arkadaşlar ben artık sizi tutmayayım" dedi ve perşembe günü maalesef örnekleri yapamadık. Ersin bize nasıl trace alınacağını falan uygulamalı olarak göstercekti, fakat çok önemli değil, çünkü bir hafta önce ben bloga güzel bir örnek koymuştum :)) Tkprof workshop diye. Hafta içinde kendim bile çok kullandım aynı kodları, zaten o zaman anladım blog'un ne olduğunu :))

10046 trace almak bir developer için o kadar da mühim değil, çünkü zaten bu onun işi olmayacak. Zaten çoğu durumda autotrace ve runstats de yeterli. Ayrıca 4. Hafta içinde performans sunumları olacak, ve bir çoğunu yine blog'dan yayınlayabilmeyi umuyorum. (En azından kendi yaptıklarımı yayınlayacağım kesin :) ) Trace ve performans değerlendirme konusundaki eksiklerimizi bir çok arkadaşım ile birlikte tamamlamış olacağız.

Biraz geyik modundayım, o yüzden uzun uzun yazıyorum. Kafamda da "CBO'nun neresinden başlasam şimdi" sorusu var. Kyte'ın kitabı güzel bir örnek bu konuda. Lewis'in kitabı ve sunumları var, oldukça etkili. Ersin'in sunumunu ve aldığım notları da açıp birşeyler yazmaya çalışacağım sizlere.

• Statement Processing 4 kısımdan oluşur: 1- Parsing, 2- Opitimization, 3- Row-source generation, 4- Execution.

1- Parsing: SQL cümlelerinin bizim anlayıp yazdığımız syntax'ından, Oracle'ın anlayıp çalıştıracağı duruma getirilmesi olarak ifade edebiliriz. Öncelikle cümle parçalara ayrılır, sonra ne tip bir operasyon olduğu belirlenir, sonra syntactic ve semantic analizi yapılır, yani gerçekten geçerli ve anlamlı bir komut olup olmadığı anlaşılır, sonra sırada shared pool check vardır. Shared pool check'in üzerinde durmak lazım biraz. Shared pool, SQL komutlarının memory'de bulunduğu alandır. SGA'dadır. SQL cümleleri shared pool'un library cache denilen bölümünde yaşar. Shared pool'un espirisi kısaca şu: Parse edilen bir SQL komutu burada bir süre saklansın, yine aynı komut çalıştırılmaya kalktığında biz aynı komut olduğunu tesbit edelim ve dolayısıyla bir daha parse edilmesinden kurtulalım. Parse maliyetli bir iştir ve dolayısıyla bu bize büyük kazanç sağlar. Evet konumuza dönelim, SQL'in anlamlı olduğu anlaşılınca shared pool check'e geldi sıra. Bunun da adımları var. Statement check, semantic check, enviroment check gibi. Kısaca, SQL cümlesi hem text olarak hem de işlev olarak tamamen aynı olduğu konusunda kontrol edilir. Eğer bu kontrolü geçemezse, statement processing'in devam eden kısımlarına uğramak zorunda kalır SQL'imiz (Optimization ve row source generation)

2- Optimization: Burası aslında en çok üzerinde durulan konu 2 gün boyunca. Optimization, sorgunun optimize edilmesi, en iyi şekilde nasıl çalıştırılacağının tesbit edilmesi ve execution plan'ının oluşturulması işlevidir. Oracle'ımızda şu anda Cost Based Optimizer(CBO) var. Eskiden Rule Based Optimizer(RBO) vardı. Nedir bunlar peki? Eskiden başlayalım :)

Rule Based Optimizer(RBO): Kurala dayalı optimizer. Optimizer'ın daha önceden belirlenmiş, sabit, katı (:P) bir çok kuralı var. Bunlara göre execution plan'ı seçiyor. Örneğin, sorguda istenen tablo üzerindeki alanda index varsa kullan, falanca tip iki tablo join ediliyorsa şu şekilde et gibi. Öncelikler belirlenmiş ve nasıl belirlendiyse öyle bir şekilde işliyor. Peki bunun kötü yanı ne? Kötü yanı, en optimum execution plan'ın duruma göre değişebilmesi. Örneğin, aynı tablo, aynı veri, aynı index üzerinde aynı sorgu çalıştırılıyor olsa bile, bazen index kullanmak daha iyi olabilirken, sorgunun farklı bir değeri için full table scan daha hızlı çalışabilir. Bazı joinleri hash join yaparsın, bazılarını duruma göre "nested loops" yaparsın. Yani bu tarz dezavantajları vardı, bizim sabit fikirli RBO'muzun :))

Cost Based Optimizer(CBO): Maliyete dayalı optimizer tamamen dinamik çalışır. Mantığı ksıaca şöyledir: Tablo üzerinde daha önce toplanmış bazı istatistiklerden yararlanarak, istenen bir sorgu için bir çok ihtimali göz önünde bulundurarak her kombinasyon için ayrı birer maliyet *tahmin* eder, ve içlerinden en az maliyete sahip olan execution plan'ı seçer. CBO, aynı sorgunun farklı zamanalrda çalıştırılması için bile farklı execution plan'ler çıkarabilir. Tamamen o anki olaya dayalı olarak, istatistikler ışığında anlık execution plan üretir. İstatistikleri yeterince güncelse(ki genelde öyledir çünkü Oracle bunalrı da çaktırmadan otomatik toplar) genellikle CBO yanılmaz, en iyi plan'ı üretir. Çok nadiren yanıldığı olabilir tabi, o zaman da direk optimizer'a sorgu içinde hint vererek o şekilde çalışmasına zorlayabiliriz. Ama bu optimizer'ı baltalamak olur ki, ne yaptığımızı, şu anda nasıl bir veriye sahip olduğumuzu, ilerde nasıl bir veriye sahip olabileceğimizi, olayın içindeki tüm obje ve faktörleri(indexler, join edilecek tablolar, içindeki veriler vs.) optimizer'dan iyi biliyor olmamız lazım.

RBO Oracle'ın çok eski versiyonlarında vardı. Daha sonra 8i ile birlikte CBO'yu geliştirdiler RBO'yu geliştirmeyi bıraktılar. Default çalışan optimizer CBO'dur. Halen geriye dönük uyumluluk açısında RBO varolmasına karşın artık RBO kullanılması tavsiye edilmiyor.

3- Row source generation: Execution Plan'ın Oracle'ın anlayacağı şekile çevrilmesi ve özel data structure'ına dönüştürülmesi işlevidir.

4- Execution: Bütün bu işlemlerden sonra execution gelir. Execution kesinlikle varolan bir adımdır. Parsing'in bir kısmı ve Optimization az önce belirttiğimiz şartlarla es geçilebilir fakat execution es geçilemez. Çalıştırılan SQL bir DDL ise veya veriyi güncelleyen bir SQL ise (update, insert delete) execution adımı operasyonun son adımıdır. Query ise (select) bir sonraki adım fetch adımıdır.

• Statement Processing konusunun espirisi, aslında CBO ile tanışmak ve parsing olayının nasıl elemine edilebileceğini anlamaktan ibaret. Bind variable'ların kullanılması bu noktada önemli.

• İkinci gün (17.07.2007) Ersin CBO konusunun ayrıntılarından ve Explain Plan'dan bahsetti biraz. Maliyetin ne olduğu konusu var önce: Maliyet, aslıdna bir çok faktörün bulunduğu matematiksel bir formül. Tahmin ettiği değerleri yerine koyuyor CBO ve tek bir rakam olarak maliyeti hesaplıyor. Sonra bu maliyetleri kıyaslayarak en iyisine ulaşıyor.

• Explain Plan, execution plan'in biraz değişik, bizim anlayabileceğimiz şekilde şekillendirilmiş halidir diyebiliriz. Explain Plan içten dışa doğru okunur. Soldan en fazla boşluğa(girintiye) sahip olan en içtedir. İç levelde olanlar önce çalıştırılır. Aynı levelde olanların ise hangi sırada çalıştırılacağı kesin değildir, zaten bu pek birşey de ifade etmez maliyet olarak. Her bir satırda yazan operasyonların ne olduğu konusunda sunumda bir miktar bilgi var, ek linkler de koyacağım links kısmına, onları uzun uzadıya burda yazmayacağım.

• Son olarak anlatılan HINT'ler var. Hint'lerin amacından yukarıda biraz bahsetmiştim. CBO'ya müdahale etmek için kullanılıyorlar. CBO'ya verilen komut gibi düşünebilirsiniz. Siz hint oalrak belirtirseniz CBO belirttiğiniz şekilde oluşturur execution plan'ı. Hint'lerle kaynakalrı da linkler kısmında belirttim, bütün hint'leri, açıklamalarını ve örneklerini bulabilirsiniz.

Ersin'in bu muhteşem iki sunumunu, çalıştırdığı demo'ları, Ertürk'ün geçen sene yapmış olduğu CBO sunumunu ve Hakkı abinin geçen sene yapmış olduğu Statement Processing sunumunu içeren rar dosyasını buradan indirebilirsiniz.

Sanırım Turkcell'de staj programına dahil olmadığı halde bu yazı dizisi takip edenler, ek kaynakalrla birlikte eğer incelemeye devam ederlerse bu sunumlara katılanlardan çok daha fazla bilgi sahibi olabilirler. Herhangi bir soru olursa bana adresinden mail atabilirsiniz, cevabına mutlaka ulaşıp yanıtlamaya çalışırım.

Herkese iyi çalışmalar.

Feed users, click here to go to this topic's page. You can find related topics and reference links and you can read the comments for this topic

Feed users, click here to post a comment for this topic

Note: There might be problems about HTML code and parsing and displaying HTML code of your viewer application. Also, there are problems caused by Google Reader on displaying of code samples on the entry. Also, the entry might be updated. So, it is recommended to visit the page of the entry.

"Kurtuluş"un hikayesi

Bilal Hatipoglu — 23.07.2007 20:20:17

“Tarihte belki de hiçbir vapur, onun kadar umutla beklenmemiş, onun kadar sevinçle karşılanmamıştı... Hiçbir vapur, tek bir yolcu taşımadığı halde binlercesini ölümden kurtarmamıştı... Tarihte belki de hiçbir vapur, onun kadar çok sevilmemişti... Ve yine hiçbir vapur, derinliklerde onun kadar kolay unutulmamıştı...”

2. DÜNYA SAVAŞINDA YUNANiSTAN

Ikinci Dünya Savasi, Avrupa'nin içlerinde bütün siddeti ile sürerken, Yunanistan ekonomik sikintilar içindeydi. Birinci Dünya Savasi'nin sonunda Anadolu topraklarinda yasanan macera, ülkeyi fazlasiyla yipratmisti. Savas sonrasinda Türkiye ile yapilan mübadele de, ülkedeki dengeleri altüst etmisti.

Bu durumda, yasanan savasta tarafsiz kalmak, Yunanistan için hayati önem tasiyordu. Ancak Naziler, Ege ve Akdeniz'de stratejik önemi olan bu küçük ülkenin bagimsiz kalmasina, daha fazla izin vermeyecekti. 28 Ekim 1940'da Italya, Yunanistan'a bir nota vererek ülkeyi isgal edeceklerini bildirdi. Kosullarin güçlügüne ragmen Yunanistan, Arnavutluk sinirinda isgale karsi topyekun bir direnis baslatti. Mussolini'nin her saldirisi siddetle geri püskürtülüyor, Italyan ordusu her geçen gün güç kaybediyordu.

Bu durum Hitler'i çileden çikarmisti. Mussolini ile yaptiklari planlara göre, o ordusunu Rus sinirina yigarken, Yunanistan Italyanlar tarafindan isgal edilecekti. Böylece yardim yollari tamamen tikanan Rusya'ya 1940 baharinda saldiri baslayacak ve operasyon kis gelmeden tamamlanacakti.

Hitler, Italyanlarin bu basarisizligina sadece 6 ay dayanabildi. 6 Nisan 1941'de iyi donanimli Alman ordulari saldiriya geçti ve sadece 20 gün içinde Yunanistan Naziler tarafindan isgal edildi.

27 Nisan 1941'de isgal tamamlanmis, Türkiye sinirina yakin bölgeler ve Ege'nin girisinde yer alan Girit Adasi, Almanlarin hakimiyetine girerken; küçük adalar ve anakara, kagit üzerinde Italyanlara verilmisti. Yönetim, General Tsolakoglu baskanliginda kurulan, kukla bir hükümete birakiliyordu.

BÜYÜK AÇLIK

Yunanistan'ın Naziler tarafından işgalinin asıl yıpratıcı etkisi, ülkenin yiyecek stoklarının yağmalanması ile başladı. Hitler, yüzyıllardır topraklarında yetişen ürünler ihtiyacını karşılamaya yetmediği için gıda ithalatı yapan Yunanistan halkının kısıtlı yiyeceğine, Rusya sınırındaki ordularını beslemek için el koymuştu.

Atinalılar, parasını ödeyerek dahi, yiyecek bulmakta zorlanmaya başlamıştı. Taşradan Atina'ya yiyecek taşınması, savaş sırasında demiryolları tahrip edildiği için durmuştu. Denizden de sevkiyat yapılamıyordu çünkü batırılan gemilerin enkazları tehlike yaratır durumdaydı. Ayrıca savaş sırasında dökülen deniz mayınları birçok bölgeye sefer yapmayı olanaksız kılıyordu.

Eldeki kısıtlı gıdanın dağıtımı karneye bağlanmıştı. Ancak bu karnelerle dağıtılan yiyecek sembolik miktardaydı. Normal bir insanin günlük tüketmesi gereken 1200 kalori iken ve asgari yaşam için 900 kalori gerekirken, devlet, karneler karşılığında ayda sadece 300 kalorilik yiyecek tahsis ediyordu. Karaborsa yaygınlaşmıştı.

Özellikle yoksul mahallelerde açlık öylesine büyüktü ki, halk, at, eşek hatta kedi, köpek eti yiyordu. 1941 sonbaharında açlıktan ilk ölümler başladı.

Bugün Yunanistan'da, açlık nedeniyle 2. Dünya Savaşı boyunca ölenlere ilişkin sayılar tartışmalıdır. BBC, 500.000 kişinin öldüğünü söylerken, Kızıl Haç bu sayıyı 250.000 kişi olarak açıklamaktadır. Ancak, Yunanlı tarihçiler bu rakamların politik kaygılar taşıdığını belirtmekte ve gerçek ölü sayısının 70.000 kişi civarında olduğunu vurgulamaktadır.

YUNANİSTAN'A YARDIM

İşgalin başladığı ilk günden itibaren Atina halkının tek umudu, dışarıdan ulaşacak bir yardımdı. Bu yardım için düşünülen ilk ülke, o zamanlar tarafsız bir konumda olan komşu Türkiye oldu.

Ancak o günlerde Türkiye'de de kıtlık yaşanıyordu. Tarımda çalışabilecek nüfusun büyük bölümü savaşın başlarında askere alınmış, yiyecek stokları, olası bir savaş ihtimaline karşı, orduyu beslemek üzere ayrılmış ve seferberlik ilan edilmişti.

Buna rağmen Türk basını, komşu ülkede yaşanan büyük açlığa duyarlı davranıyor, gelen haberleri kamuoyuna titizlikle yansıtıyordu. Türk halkı da konuya aynı duyarlılıkta yaklaşıyor, kısıtlı imkanlarına rağmen, komşu ülkeye yardım etmek için elinden geleni yapmak istiyordu.

Sonunda Cumhurbaşkanı İsmet İnönü, 19 yıl önce topraklarından attığı düşman ordusunun halkına yardım etmek için alınan karara imza attı. Yunanistan'a dostluk elini uzatan ilk ülke Türkiye olacak, Kızılay bu işe aracılık edecekti. Türk Hükümeti, savaşın sonuna kadar, Yunanistan'a 50.000 ton gıdayı göndermeyi taahhüt etmişti.

Kampanya, Kızılay tarafından yürütülecekti ancak Gümrük Bakanlığı, Ticaret Bakanlığı ve Dışişleri Bakanlığı temsilcilerinden oluşturulan bir komisyonun, gönderilecek malzemenin içeriğini onaylaması uygun görülmüştü. Kampanyaya İngiliz Kızıl haçı ve Amerika'da yaşayan Rumların kurduğu dernekler de maddi destek verecekti.

Kısıtlı imkanlara rağmen, kampanya yurt çapında büyük ilgi gördü. Yardım yapmak isteyenler bu taleplerini komisyona bildiriyor, onaylanan yardımlar, uluslararası Kızılhaç kurallarına göre paketlenerek yolculuğa hazırlanıyordu.

Devletin elinde bu yardımları taşıyabilecek kapasitede bir gemi bulunmadığından, Kızılay, yardımları taşımak için özel sektörün elindeki vapurlar arasından, kış aylarında açık deniz seyrine dayanabilecek durumda olanları tespit etti.

Sonunda Tavilzade Biraderler şirketinin elindeki 2400 tonluk kuru yük gemisi uygun bulundu. 1882 yılı yapımı gemi, ilginç bir rastlantı sonucu 19 yıl önce Yunanistan ile yaşanan savaşla aynı adı taşımaktaydı: KURTULUŞ

Kurtuluş Vapuru, dört tarafına Kızılay amblemleri konularak sefere hazırlandı. Bu amblemler Kurtuluş'u, seyir esnasında savaş uçakları ve denizaltılar tarafından saldırıya uğramaktan koruyacaktı. Kurtuluş Vapuru'nun izleyeceği rotanın belirlenmesi de başka bir sorundu. Savaş öncesinde Ege'den geçen ticaret rotalarına, işgal sırasında dökülen mayınlar, seyir için büyük tehlike oluşturuyordu. Bu nedenle vapurun her seferi için Yunan, Alman, İtalyan ve İngiliz hükümetleri ile yazışmalar yapılıyor, Kurtuluş'un rotası için onay alınıyordu.

İlk seferi için 13 Ekim 1941'de, Karaköy rıhtımından yola çıkan Kurtuluş Vapuru mürettebatı, durumun dehşetini daha Pire Limanı'na girerken fark etmişti. Limanın içi, bombalanma sırasında burada bulunan gemiler batırıldığından bir gemi mezarlığını andırıyordu. Kurtuluş, batıkların arasından güçlükle kıyıya yanaştı. Vapurun yiyecek getireceği haberini alan binlerce Atinalı, saatlerdir onun gelişini bekliyordu.

Yardımlar, Kızılhaç tarafından, hastanelerden başlamak üzere, önceden belirlenmiş listeye göre dağıtıldı. Bu çalışmalara refakat eden Alman ve İtalyan askerleri ise, Türk delegeleri ve Kızılhaç görevlilerine nazik davranıyordu.

Mürettebat gördüğü manzaradan o kadar etkilenmişti ki dönüş için kumanya olarak ayırdıkları yiyecekleri limana bıraktılar.

Kurtuluş Vapuru, Pire Limanı'na her biri diğerinden daha acılı öykülerin yaşandığı üç sefer daha yaptı ve Şubat 1942'ye kadar bu 4 seferde yaklaşık 7.100 ton gıdayı Yunan halkına ulaştırdı. Bu yardımlar, elbette sorunun çözümü için yeterli değildi. Ancak Yunan halkı, efsaneye dönüşen Kurtuluş Vapuru sayesinde, umutla beklemeyi ve gelen yardımları paylaşmayı öğrendi.

Kurtuluş Vapuru, 20 Şubat 1942'de saat 9.15'de Marmara Adası'nın kuzey kayalıklarında, yaklaşık 2000 ton gıda ile sulara gömüldü. Kurtuluş'un 36 kişilik mürettebatı kazadan sağ kurtulmayı başardı.

BARIŞI TAŞIYAN VAPUR NASIL HAZIRLANDI?

Kurtuluş Vapuru ilk kez 2004 yılında TRT 2 de yayınlanan "Tarihte Gezintiler" programında karşımıza çıktı. Gazete arşivlerinden hazırlanan 4 dakikalık haberde, 2. Dünya Savaşı sırasında Yunanistan'a gıda yardımını taşırken batan bir Türk vapuru anlatılıyordu. Program bittiğinde şaşkına dönmüştüm. Yıllardır diplomatik sorunlar yaşadığımız bir halka neden yardım eli uzatmıştık? Vapur neden "Kurtuluş" adını taşıyordu? Böyle bir öykü neden yıllardır gündeme gelmemişti? Kurtuluş Vapuru'nun enkazı neredeydi? Olayın Türk ve Yunanlı tanıkları yaşıyor muydu?

Öyküyü ve sorularımı ekibimle paylaştım. Kurtuluş Vapuru'nun öyküsü herkesin büyük ilgisini çekmişti. Olay, belgesel yapılmalıydı.

İlk aşama gazete arşivlerinde Kurtuluş'un izini bulmaktı. Araştırmacılarımızın ilk getirdiği kupürler hayret vericiydi. Tüm gazeteler Yunanistan'a yapılan yardım çalışmalarına geniş yer ayırmış, vapurun her seferi, büyük ilgi görmüştü.

Ancak araştırmayı derinleştirdiğimizde, döneme ait resmi kayıtların büyük bölümünün imha edildiğini, yardım çalışmalarını organize eden Kızılay'ın arşivlerinde dahi Kurtuluş'a ait belge bulunmadığını tespit ettik.

Üstelik batığın bulunduğu yer, hiçbir resmi kaynakta geçmiyor, eldeki Lloyd kaydı gazete arşivlerindeki bilgilerle çelişiyordu. Yunanistan'da da durum farklı değildi. Kurtuluş Vapuru'nun seferleri sırasında ülke işgal altında olduğundan, döneme ait yazılı kaynaklara ulaşmak mümkün değildi.

Ankara, İstanbul ve Atina'da gazete arşivleri, kütüphaneler ve resmi kaynaklar arasında yaklaşık bir yıl süren bu çalışma, Kurtuluş Vapuru hakkındaki detaylı bilgilere ulaşmamızı sağladı.

Ancak, batığın bulunduğu yerin tespit edilebilmesi, uzmanlık ve teknoloji gerektiren bir işti. Bu konuda Türkiye'nin uluslararası düzeyde en önemli batık araştırmacısı olan Selçuk Kolay'dan yardım istedik. Selçuk Kolay, araştırmayı bizzat yönetmeyi ve teknoloji harikası teknesi Bothnia ile Kurtuluş'un yerini tespit etmeyi memnuniyetle kabul etti.

Diğer yandan keşif ekibimiz Marmara Adası'ndan iyi haberler getirmişti. Adanın yaşlı balıkçıları, Kurtuluş adlı bir geminin kuzey kayalıklarında battığını hatırlıyordu. Anlatılanlar, gazete arşivlerindeki batış öyküsünü doğrular nitelikteydi.

Sonunda, Bothnia ile bölgeye hareket ettik. Marmara Adası'nın kuzeyinde yer alan Saraylar köyündeki balıkçılar, Kurtuluş'un öyküsünü bilmese de enkazının yerini biliyorlardı. Ancak adalı balıkçılar, batığın 60'lı yıllarda, hurdacılar tarafından söküme uğradığını da söylüyorlardı.

Kurtuluş'un son fotoğrafında görünen kayalıklar, balıkçıların gösterdiği yer ile benzerlik taşıyordu. Yan tarama sonar cihazı ile yapılan ölçümler, batığın Kurtuluş olduğunu doğruladı. Daha sonra dalış yaparak, batığın görüntülerini günışığına çıkardık.

Kurtuluş'un gerçek batış yeri artık tespit edilmişti, ancak vapuru Yunanistan'da hatırlıyorlar mıydı? Olayın canlı tanıklarına ulaşmak ve merak ettiğimiz diğer soruların yanıtlarını bulmak için Pire ve Atina'ya gittik.

Atina'da 2. Dünya Savaşı'nı yaşayanların tümü Kurtuluş'u hatırlıyordu. Çekimler sırasında bize büyük konukseverlik gösterdiler. Röportaj yaptığımız herkes, konuşmasını, Yunanistan ve Türkiye arasındaki gerginliğin anlamsızlığını vurgulayarak bitiriyordu.

Tüm çalışma boyunca tek üzüntümüz, öykünün Türkiye'deki kahramanlarının hayata veda etmiş olmasıydı.

Tarihçi Georgeos Margaritis'in söyledikleri Kurtuluş efsanesini aydınlatıyordu: "Kurtuluş, Şubat 1942'ye kadar Yunanistan'ın en zor dönemlerinde yaptığı seferlerle Yunan halkı için umudun sembolü oldu. O battıktan sonra da Türkiye'den gemiler geldi ancak bütün o gemilerin adı Kurtuluş olarak kaldı.

KURTULUŞ DERİNLİKLERDE

Yunanistan'da görüştüğümüz canlı tanıkların tamamı, Kurtuluş'un Alman mayını ya da denizaltı tarafından batırıldığı kanısındaydı.

Ancak bugün talihsiz vapurun derinliklerdeki görüntüleri, saldırı sonucu infilak eden gemilerden daha kötü durumdaydı. Batık, 60'lı yıllarda metal bölümlerini yeniden kullanmak isteyen Hurdacılar tarafından dinamitle parçalanmıştı.

Sualtında futbol sahası büyüklüğünde alana yayılan enkazın sadece iskelet kısmı ayaktaydı.

Yağmalanmış da olsa, Kurtuluş Vapuru'nun batış nedeni ve yeri artık biliniyor. Sivil denizcilik tarihinde bu önemli vapur bir daha unutulmayacak...

Yukarıdaki hikayenin bir benzerini bu haftasonu seçim için memleketim Bursa'ya giderken ido feribotu'nda ido'nun çıkardığı sealife dergisinde okudum. Oldukça etkilendim. Daha sonra yaptiğim araştırmada yakın zamanda bunun belgelselinin çekildiğini öğrendim. Öncelikle bu konuda emeği geçenlere tebriklerimi ve teşekkürlerimi sunuyorum. İlgili kaynaklara aşağıda linklerden ulaşabilirsiniz.

Biz tarihimizi bilmezsek bizi kandırırlar. Siz Türkler barbarsınız, soykırım yaptınız derler. İşte biz, bu gibi örneklerle dolu şanlı tarihimizi iyi bilmeliyiz. Ama bize lisede, üniversitede tarih derslerinde bunlar okutulmaz. Neden okutulmaz diye araştırırsanız suçlu bile çıkabilirsiniz. Bize "humanities" gibi dersler okutur. Bu tür haberler basında da yer almaz. Bunlarin hepsini sadece büyük hatalardan, tesadüflerden ibaret sanan sadece kendini kandırır. Bu, Türkiye üzerinde 3 asırdan beri oynanan bir oyunun sadece küçük bir parçası…

Yukarıdaki yazıyı bir kez daha okumalı, tanıdıklarımızla paylaşmalıyız. Bu tür örnekleri her alanda dile getirmeliyiz. En önemlisi bize unutturulmaya çalışılan şanlı tarihimizi iyi öğrenmek ve bizden sonra gelen nesillere aktarmak için elimizden gelen çabayı sarfetmeliyiz.

Feed users, click here to go to this topic's page. You can find related topics and reference links and you can read the comments for this topic

Feed users, click here to post a comment for this topic

Note: There might be problems about HTML code and parsing and displaying HTML code of your viewer application. Also, there are problems caused by Google Reader on displaying of code samples on the entry. Also, the entry might be updated. So, it is recommended to visit the page of the entry.

Turkcell Staj Günlüğü - 7: Concurrency and Consistency

Bilal Hatipoglu — 20.07.2007 08:51:05

Merhaba, günlük serisinin gecikmeli 7. yazısında stajımızın 2. haftasının kalan 2 gününden bahsedeceğim. Baya yoğun bir hafta geçirmekte olduğum ve performans konusu üzerinde çok uğraştığım için biraz gecikmeli oldu bu.

Perşembe günü (12.07.2007) TUrkcell Akademi - İstiklal Cad. binasında staj oryantasyonu vardı. Farklı departmanlardan bir çok stajyer katıldı bu programa. Tüm gün bounca çeşitli sunumlar yapıldı. Sunumlar çok fazla konumuzla alakalı olmadığı için bunları es geçiyorum :)

Cuma günü Ertürk'ün muhteşem "Concurrency and Consistency" sunumu vardı. Güzel bir sunumdu gerçekten. Oracle'ı Oracle yapan mekanizması anlatıldı. Ertürk'ün sunumunu buradan indirebilirsiniz.

Sunum biraz teoride kaldı, pek pratik şansı olmadı maalesef. Fakat en kısa zamanda telafi olarak workshop da yapılacak bu konuda. Vakit bulursam workshop'tan önce veya sonra blog'da bir makale de hazırlayabilirm bu konuda.

Her makaleden önce tek sunudan ziyade bir çok diğer kaynağı da gözden geçiriyorum, bu yüzden zaman alıyor. O yüzden bu makalenin yayınlanması bir hafta gecikti, kusura bakmazsınız heralde :)

Burada elime geçirebildiğim başka sunumları ve ek kaynakları da yayınlamaya çalışıyorum. Geçen senenin staj döneminde, Hüsnü'ün yapmış olduğu "Locks, concurrency and consistency" konulu sunum da ek kaynak olarak gözden geçirmeye değer. Hüsnü'nün sunumlarını da buradan indirebilirsiniz.

Ben bu konudaki bazı notlarını aktaracağım. Sunumları da inceleyerek mutlaka takip edin.

• Lock yapısı database'leri diğer database'lerden ve database'leri normal dosya sisteminden ayırt eden yapıdır. Transaction ve lock yapısı bir database'i database yapan yapıdır aslında. Bİr çok şekilde implement edilebileceği için farklı databse'ler arasında en çok farklılk gösteren yapılardır bunlar.

• Oracle'ın lock yapısı çok başarılı gerçekten. Oracle'ı Oracle yapan da bu. Row bazında kilitlemesi, Lock escalation olmaması, Dirty read olmaması, ne olursa olsun sistemin hep consistent state'de kalması, okuyucuların yazıcıları ve yazıcıların okuyucuları beklememesi... bunlar hep Oracle'ın mükemmel özellikleri. Çoğu diğer database'lerde olmayan özellikler bunlar.

• Oracle, kayıtları row bazında kilitler. Yani bir row (satır) üzerinde değişiklik yapılırken Oracle sadece o satırı kilitler. Bu, tablonun diğer row'larını etkilemez. Bu kilit yazma kilididir, okuyucular aynı row'ı bile okuyabilirler, kesinlikle yazıcıları beklemezler. Diğer database'lerin tablo bazında, page bazında vs. tarzı değişik implementasyonları var. Bazıları da önce row bazında kilitlemesine rağmen, çok fazla referans geldiğinde kilidi page bazına çıkarma, daha sonra tablo bazına çıkarma vs. gibi şeyler yapıyorlar. Buna lock escalation diyoruz. Oracle'da lock escalation yok.

• Lost update denen bir durum var. Şöyle ki: User1 bir row'u okudu, sonra User2 okudu. User1 ona göre işlemini yaptı, commit de etti, fakat user2 bunun farkında değil, o da işlemini yapıp commit etti. Bir örnekle anlatalım: Online satış sistemimiz olsun. Brinci müşteri geldi, ürüne tıkladı, ürünü inceliyor. Üründen stokta bir tane var ve müşteri de stokta var diye görüyor. Sonra ikinci kullanıcı da aynı ürüne bastı ve aynı ürünü inceliyor, o da stokta var görüyor. SOnra birinci kullanıcı "satın al"a bastı ve stoktaki tek ürünü satın aldı, stok miktarında düşüldü. Sonra ikinci kullanıcı da satın almaya karar verdi ve "satın al"a bastı. Stok miktarı -1'e inemeyeceğine ve olmayan ürünü müşteriye satamayacağımıza göre İşte bu durumun kontrol altına alınması gerekiyor. Lost update durumunu kontrol etmek için iki tarz locking mekanizması var:

1- Pessimistic Locking: Birinci kullanıcı row'a eriştiği an kilit konur ve ikinci kullanıcının ulaşması engellenir. Bu, concurrency'i çok düşürür. Ayrıca http stateless bir bağlantı olduğu için bunun implement edilmesi mümkün değil. Oracle'da "SELECT FOR UPDATE" komutu ile bu şekilde select edildiği an kilit oluşturulabilir.

2- Optimistic Locking: Select edildiğinde kayıt kilitlenmez, fakat update edilmeye çalışırken bir şekilde durum tekrar kontrol edilerek durum handle edilir. Oracle'ın default kilitleme mekanizması zaten optimistic, yani normalde zaten okuyucular hiçbir zaman kayıtları kilitlemez. Tek istisnası, pessimistic locking kullanan "SELECT FOR UPDATE" komutu.

Google'a "Pessimistic Locking" yazınca ilk çıkan sayfada zaten güzel bir şekilde anlatılmış bu, kafanıza takıldıysa biraz da burdan okuyabilirsiniz: http://www.agiledata.org/essays/concurrencyControl.html

Optimistic Locking'in çeşitli implementasyonları var.

a) Version Column: Kayıda ayrı bir versiyon kolonu eklenir, buna örneği en son update zamanı koyulur. Böylece yukarıdaki örnekte ikinci müşteri ürünü almaya çalışırken bu kolon kontrol edilerek data'nın değiştiği anlaşılır.

b) Hashing: Data okunduğunda hash'lenir. Update edilirken tablodaki datanın tekrar hash'i alınır, hash'ler farklı çıakrsa data değişmiş demektir.

c) ORA_ROWSCN: Bu oracle'ın 10g versiyonunda geldi. Oracle, her row'a özel bir ROWSCN değeri atar, her update'de bu değer otomatik oalrak değiştirilir. Dikkat ederseniz yukarıdaki iki yöntemi sistemi tasarlayan ve kodlayan kendisi yapıyor, ekstra bir kolon ekliyor veya hash alma kodunu yazıyor. Burada bu işi oracle kendisi yapıyor.

• Block ve Deadlock, adı üstünde, kilitli bir row'un ona referans eden yazıcıları bloklaması demek, deadlock da farklı kaynaklar üzerine kilit koyan iki (veya circular olarak daha fazla) job'ın birbirini beklemesi demek. Oracle deadlock durumlarını tesbit ederek hata döndürür, son yapılan komutun etkilerini de geri alır. (Statement level rollback) Zaten Oracle gibi çok iyi concurrent ve consistent bir sistemde çok nadir görülür.

• Transaction lock'ları datablock'lar içinde saklanır. Transaction ilk değiştirilen veri ile başlar, commit veya rollback edildiğinde sonlanır. Transactionların aktif olup olmadıkları kontrol edilir, hatalı transaction'lar kapatılarak kilitleri sisteme geri bırakılır.

• Latch, lock'lardan daha basit, daha hafif bir birimdir. Lock'lar gibiişlev görür. İşlemcilerin "Test and set", "Load and clear", "Compare and swap" gibi atomik operasyonlarıyla implement edilir. Latch'ler üzerinde bekleyen process'ler bir süre spin ederek busy wait yaparlar. Daha sonra uyutularak belli periyotlarda uyandırılırlar. Bundan PMON process'i sorumludur.

• Latch ve Lockların fazla olması consistency'i bozar ve wait'lerin artmasına neden olur. Biz bunu istemeyiz. DBMS'lerin amaçlarından bir tanesi de minimum sayıda latch ve lock kullanmaktır.

• Oracle, internal lock'lar dışında, "select for update..", "lock table..." gibi sorgular ve DBMS_LOCK paketiyle external olarak da kilit koyulmasına olanak tanır. Fakat consistency bozualacağı için bunları dikkatli kullanmak gerekir. Normal şartlarda Oracle concurrency'i korumak için gerekli kilitleri içinde zaten oluşturur.

• Dirty read, commit edilmemiş data'nın okunması demektir. Oracle dirty read'e izin vermez, okuyucular data değiştirilmiş olsa bile undo block'lardan en son consistent data'yı okur.

• Non-Repeatable (fuzzy) read, bir transaction içinde aynı select sorgusunu birden çok kez çağırdığımızda farklı sonuçların dönmesidir. Bu, çok kullanıcılı sistemlerde muhtemeldir, A transaction'u bir select çalıştırdıktan sonra B Transaction'ı o row'ları update'liyerek commit ederse, A Transaction'u tekrar aynı sorugyu çalıştırdığında farklı sonuç alacaktır.

• Phantom read de, non-repeatable read gibi, fakat burada insert söz konusu. Transaction A where sözcüğüyle bir select çalıştırdıktan sonra B Transaction'u o where şartını sağlayan bir insert yaparsa, A aynı sorguyu tekrar çalıştırdığında farklı sonuç alacaktır.

Bu konudaki notlarım da bu kadar. Bu da önemli ,ayırt edici bir konuydu. Anlamakta güçlük çektiğiniz bir şey varsa Concepts guide'ın ilgili bölümü'nden bir göz atın önce, sonra Kyte'ın kitaplarından pekiştirebilirsiniz.

Feed users, click here to go to this topic's page. You can find related topics and reference links and you can read the comments for this topic

Feed users, click here to post a comment for this topic

Note: There might be problems about HTML code and parsing and displaying HTML code of your viewer application. Also, there are problems caused by Google Reader on displaying of code samples on the entry. Also, the entry might be updated. So, it is recommended to visit the page of the entry.

Derin anlamlı sözler - Bunlar da Türkçe olanlar :)

Bilal Hatipoglu — 19.07.2007 15:58:53

Bunlar da Türkçe. Ezberlenesi sözler gerçekten. İngilizce olanlara buradan ulaşabilirsiniz. Bunlar ingilizce olanlardan farklı, inglizce bilenler onlara da göz atsın.

• Her kim ki gün boyunca bir arı kadar aktif, bir boğa kadar güçlü, bir at kadar çalışkan olduğu halde, akşam olunca bir köpek kadar bitkin eve dönüyorsa bir veterinere görünmelidir. Çünkü eşşek olması, kuvvetle muhtemeldir. Chang Ying Yue

• İkincilik istatistiklere bir kayıttır. Tarih şampiyonları alkışlar. Carl Lewis

• İnsanlara en adil şekilde dağıtılan nimet akıldır. Çünkü kimse aklından şikayetçi değildir. Montaigne

• Dünyanın en güç işi, bir şeyin nasıl yapılacağını bilirken, başka birinin nasıl yapamadığını ses çıkarmadan izlemektir. Juspe Kunde

• Bilgi güçtür. Francis Bacon

• Büyük kafalar fikirleri, orta kafalar olayları, küçük kafalar ise kişileri konuşur. Hyman G. Rickover

• Her zaman doğruyu söyle hiçbir zaman ne dediğini hatırlamak zorunda kalmazsın. Mark Twain

• Oturarak başarıya ulaşan tek varlık tavuktur. H. J. Brown

• Okumak, insanı dolu yapar. Konuşmak, hazırcevap yapar. Yazmak da ince düşünceli yapar. Hepsine sahip olan insan, bulunmaz insandır.

• Yeni fikirleri yakalamaktan daha zor olan şey, beynimizin tüm köşelerini tutmuş olan eski fikirlerden kurtulmaktır. John Maynard Keynes

• Küçük şeylere gereğinden çok önem verenler, elinden büyük iş gelmeyenlerdir. Eflatun

• Bir yerde küçük insanların büyük gölgeleri oluşuyorsa orada güneş batıyor demektir. Nietzsche

• Gideceğiniz yeri bilmiyorsanız, vardığınız yerin önemi yoktur. p. Drucker

Bu da sonuncusu :)

• Çok söz hamal yüküdür. Yunus Emre

Feed users, click here to go to this topic's page. You can find related topics and reference links and you can read the comments for this topic

Feed users, click here to post a comment for this topic

Note: There might be problems about HTML code and parsing and displaying HTML code of your viewer application. Also, there are problems caused by Google Reader on displaying of code samples on the entry. Also, the entry might be updated. So, it is recommended to visit the page of the entry.

Some favourite quotes

Bilal Hatipoglu — 19.07.2007 15:57:05

Some favourite quotes. Really favourable, worth memorizing. They are in mixed order. Comment on them, select the best or submit some.

Even many of them seems non-technical or doesn't make sense on the first look, but I really can have a connection for all of them wıth my professional life, Orace, CSE or beyond:

• The strongest enemy will hide in the last place you would ever look. Julius Caesar

• The journey of a thousand miles begins with one step. Lao-Tse

• Judge people less on their mistakes than on how they handle their mistakes. Ron Hall

• There are many hypotheses in science which are wrong. That's perfectly all right; they're the aperture to finding out what's right. Carl Sagan

• Men always want to be a woman's first love - women like to be a man's last romance. Oscar Wilde

• People fear what they do not understand. Bruce Lee

• Lift your face towards the sun - then you will not see the shadows. Helen Keller

• Experience is not what happens to you, it is what you do with what happens to you. Aldous Huxley

• Only two things are infinite, the universe and human stupidity, and I'm not sure about the former. Albert Einstein

• Wise men don't need advice. Fools don't take it. Benjamin Franklin

• Minds are like parachutes - they only function when open. Thomas Dewar

• The best way to speed something up is not to do it at all. Cary Millsap

• If your happiness depends on what others do you really have got a problem. Richard Bach

• Anyone can become angry - that is easy, but to be angry with the right person, to the right degree, at the right time, for the right purpose, and in the right way - that is not easy. Aristotle

• Nothing is more dangerous than an opinion if that is the only one you got. Emile Chartier

• If you get upset about small things, you are probably not much larger than them. Alf Henrikson

• All generalizations are dangerous, even this one. Alexandre Dumas fils

• Never tear down a fence until you know why it was raised. Robert Frost

• Good timber does not grow under comfortable circumstances, the stronger the wind, the stronger the tree. J. Willard Marriott

• Formal Education will make you a living; Self education will make you a fortune. Jim Rohn

• Wise men talk because they have something to say; fools, because they have to say something. Plato

• Real knowledge is to know the extent of one's ignorance. Confucius

• A man should look for what is, and not for what he thinks should be. Albert Einstein

• I do not know with what weapons World War 3 will be fought, but World War 4 will be fought with sticks and stones. Albert Einstein

• If we knew what it was we were doing, it would not be called research, would it? Albert Einstein

• If you can't explain it simply, you don't understand it well enough. Albert Einstein

• It's not that I'm so smart, it's just that I stay with problems longer. Albert Einstein

• No amount of experimentation can ever prove me right; a single experiment can prove me wrong. Albert Einstein

• Once we accept our limits, we go beyond them. Albert Einstein

• Perfection of means and confusion of ends seem to characterize our age. Albert Einstein

• The environment is everything that isn't me. Albert Einstein

• The hardest thing to understand in the world is the income tax. Albert Einstein

• The most incomprehensible thing about the world is that it is comprehensible. Albert Einstein

• The pioneers of a warless world are the young men (and women) who refuse military service. Albert Einstein

• The release of atomic energy has not created a new problem. It has merely made more urgent the necessity of solving an existing one. Albert Einstein

• The secret to creativity is knowing how to hide your sources. Albert Einstein

• The world is a dangerous place to live; not because of the people who are evil, but because of the people who don't do anything about it. Albert Einstein

• There comes a time when the mind takes a higher plane of knowledge but can never prove how it got there. Albert Einstein

• Things should be made as simple as possible, but not any simpler. Albert Einstein

• We can't solve problems by using the same kind of thinking we used when we created them. Albert Einstein

• When you are courting a nice girl an hour seems like a second. When you sit on a red-hot cinder a second seems like an hour. That's relativity. Albert Einstein

Questioning is an important thing. Asking a lot of questions means one of the folloving: 1- You really understand or about to understand. 2- You understand nothing. Both cases are good, first case is OK, second case is also OK because you will have an additional chance to understand with asking questions or just telling that you understand nothing.

Some quotes about questioning:

• The important thing is not to stop questioning. Curiosity has its own reason for existing. One cannot help but be in awe when he contemplates the mysteries of eternity, of life, of the marvelous structure of reality. It is enough if one tries merely to comprehend a little of this mystery every day. Never lose a holy curiosity. Albert Einstein

• It is better to ask and appear ignorant than to remain ignorant. Chinese adage

• It is better to know some of the questions than all of the answers. James Thurber (1894 - 1961)

• The most erroneous stories are those we think we know best - and therefore never scrutinize or question. Stephen Jay Gould (1941 - 2002)

• The outcome of any serious research can only be to make two questions grow where only one grew before. Thorstein Veblen (1857 - 1929)

• Judge of a man by his questions rather than by his answers. Voltaire (1694 - 1778)

Predicting future is important, especially on database design. Even the most professional people cannot predict the developement on technology. It is really hard.

Some example quotes from past:

• Well informed people know it is impossible to transmit the voice over wires and that were it possible to do so, the thing would be of no practical value. Editorial in the Boston Post (1865)

• There is no need for any individual to have a computer in their home. Ken Olson, 1977, President, Digital Equipment Corp.

• Atomic energy might be as good as our present-day explosives, but it is unlikely to produce anything very much more dangerous. Winston Churchill (1939)

• I think there is a world market for maybe five computers. Thomas Watson, chairman of IBM, 1943

• Where a calculator on the ENIAC is equipped with 19,000 vacuum tubes and weighs 30 tons, computers in the future may have only 1,000 vacuum tubes and perhaps only weigh 1.5 tons. Popular Mechanics, March 1949

• There are risks and costs to a programme of action but they are far less than the long range risks and costs of comfortable inaction. John. F. Kennedy

• 640Kb ought to be enough for anyone. Bill Gates - 1981

Feed users, click here to go to this topic's page. You can find related topics and reference links and you can read the comments for this topic

Feed users, click here to post a comment for this topic

Note: There might be problems about HTML code and parsing and displaying HTML code of your viewer application. Also, there are problems caused by Google Reader on displaying of code samples on the entry. Also, the entry might be updated. So, it is recommended to visit the page of the entry.

Oracle performance analysis - Tkprof workshop

Bilal Hatipoglu — 17.07.2007 15:54:23

This entry has examples about how to gather SQL statement statistics with 10046 SQL Trace and TKPROF. Please first read the entry about the SQL Performance analysis: Oracle performance analysis - Tracing and performance evaluation

Note: I have written the code on a fresh install Oracle 10gR2 XE, so that you can follow step by step and see the errors and solutions. You may get different errors for other versions.

Version: Oracle XE Release 10.2.0.1.0
Server: Suse Linux 9.2 on VMWare Workstation 5.5.3
Client: SQL*Plus: Release 10.2.0.1.0 Console

Some parameters about code:

SYSDBA user & password: sys/1234
Test user & password: test/test
Server machine name: linux

SQL> conn sys/1234@XE as sysdba
Connected.
SQL> drop user test cascade;

User dropped.

SQL> create user test identified by test;

User created.

SQL> grant connect, resource, plustrace to test;

Grant succeeded.

SQL> grant alter session to test;

Grant succeeded.

SQL> grant all on plan_table to test;

Grant succeeded.

SQL> grant select on v_$session to test;

Grant succeeded.

SQL> grant select on v_$process to test;

Grant succeeded.

SQL> grant select on v_$parameter to test;

Grant succeeded.

SQL> conn test/test@XE
Connected.
SQL> alter session set timed_statistics = true;

Session altered.

SQL> alter session set max_dump_file_size = unlimited;

Session altered.

SQL> alter session set tracefile_identifier = 'test_1';

Session altered.

SQL> CREATE TABLE sql_trace_test (
  2    id  NUMBER,
  3    description  VARCHAR2(50)
  4  );

Table created.

SQL> CREATE OR REPLACE PROCEDURE populate_sql_trace_test (p_loops  IN  NUMBER) AS
  2    l_number  NUMBER;
  3  BEGIN
  4    FOR i IN 1 .. p_loops LOOP
  5      INSERT INTO sql_trace_test (id, description)
  6      VALUES (i, 'Description for ' || i);
  7    END LOOP;
  8  
  9    SELECT COUNT(*)
 10    INTO   l_number
 11    FROM   sql_trace_test;
 12  
 13    COMMIT;
 14  
 15    DBMS_OUTPUT.put_line(l_number || ' rows inserted.');
 16  END;
 17  /

Procedure created.

SQL> alter session set events '10046 trace name context forever, level 8';

Session altered.

SQL> EXEC populate_sql_trace_test(p_loops => 5);

PL/SQL procedure successfully completed.

SQL> select * from dual;

D                                                                               
-                                                                               
X                                                                               

SQL> alter session set events '10046 trace name context off';

Session altered.

SQL> 
SQL> SELECT s.sid,
  2         s.serial#,
  3         pa.value || '/' || LOWER(SYS_CONTEXT('userenv','instance_name')) ||
  4         '_ora_' || p.spid || '.trc' AS trace_file
  5  FROM   v$session s,
  6         v$process p,
  7         v$parameter pa
  8  WHERE  pa.name = 'user_dump_dest'
  9  AND    s.paddr = p.addr
 10  AND    s.audsid = SYS_CONTEXT('USERENV', 'SESSIONID');

       SID    SERIAL#                                                           
---------- ----------                                                           
TRACE_FILE                                                                      
--------------------------------------------------------------------------------
        25        214                                                           
C:\ORACLEXE\APP\ORACLE\ADMIN\XE\UDUMP/xe_ora_3384.trc                           
                                                                                

SQL> 
SQL> quit
Disconnected from Oracle Database 10g Express Edition Release 10.2.0.1.0 - Production

C:\Documents and Settings\stbhatipoglu>cd C:\ORACLEXE\APP\ORACLE\ADMIN\XE\UDUMP/

C:\oraclexe\app\oracle\admin\XE\udump>dir
 Volume in drive C is System
 Volume Serial Number is 0482-2B10

 Directory of C:\oraclexe\app\oracle\admin\XE\udump

17.07.2007  11:51              .
17.07.2007  11:51              ..
11.07.2007  13:22               725 xe_ora_1116.trc
11.07.2007  13:20               638 xe_ora_1624.trc
11.07.2007  13:21             2.218 xe_ora_2108.trc
13.07.2007  09:36               855 xe_ora_236.trc
14.07.2007  13:13               667 xe_ora_2540.trc
13.07.2007  09:29               857 xe_ora_2588.trc
13.07.2007  09:49               638 xe_ora_2632.trc
17.07.2007  10:26             1.784 xe_ora_2680.trc
11.07.2007  13:22               638 xe_ora_2736.trc
13.07.2007  09:30               638 xe_ora_2756.trc
13.07.2007  09:24               754 xe_ora_2828.trc
14.07.2007  13:13               725 xe_ora_2860.trc
17.07.2007  11:52           202.448 xe_ora_3384_test_1.trc
13.07.2007  09:49               857 xe_ora_3460.trc
11.07.2007  13:22             1.852 xe_ora_3512.trc
13.07.2007  09:29               638 xe_ora_3636.trc
17.07.2007  11:50               735 xe_ora_3760.trc
11.07.2007  15:47                47 xe_ora_3796_test_1.trc
16.07.2007  16:39             2.103 xe_ora_3964.trc
13.07.2007  09:30               725 xe_ora_4580.trc
11.07.2007  13:21               990 xe_ora_4632.trc
11.07.2007  15:43             1.668 xe_ora_4888.trc
13.07.2007  09:21               667 xe_ora_5352.trc
13.07.2007  09:36               638 xe_ora_5696.trc
11.07.2007  15:50           115.593 xe_ora_5700.trc
11.07.2007  13:21               638 xe_ora_5808.trc
13.07.2007  09:28             4.682 xe_ora_5848.trc
13.07.2007  09:49               725 xe_ora_6016.trc
16.07.2007  11:26             1.772 xe_ora_6248.trc
16.07.2007  12:01               741 xe_ora_6680.trc
13.07.2007  09:18               858 xe_ora_840.trc
16.07.2007  16:39               666 xe_ora_908.trc
              32 File(s)        350.180 bytes
               2 Dir(s)   7.933.313.024 bytes free

C:\oraclexe\app\oracle\admin\XE\udump>tkprof xe_ora_3384_test_1.trc d:\trcout.txt explai
n=test/test waits=yes sys=yes

TKPROF: Release 10.2.0.1.0 - Production on Sal Tem 17 11:54:24 2007

Copyright (c) 1982, 2005, Oracle.  All rights reserved.



C:\oraclexe\app\oracle\admin\XE\udump>

Download Code

You can download my tkprof output file from here. Summary

First, we have to create "plustrace" role and "plan_table" table, with their scripts, so I have run "@$ORACLE_HOME/sqlplus/admin/plustrce.sql;" and "@$ORACLE_HOME/rdbms/admin/utlxplan.sql" scripts on server. If you are using Oracle XE on local windows machine, instead, you must use: "@C:\oraclexe\app\oracle\product\10.2.0\server\sqlplus\admin\plustrce.sql" and "@C:\oraclexe\app\oracle\product\10.2.0\server\rdbms\admin\utlxplan.sql".

Then, we created our test user with required priviliges that are, plustrace role and grant to plan_table, v_$session, v_$process and v_$parameter. Then, we login from our user and start coding.

We created a table called "sql_trace_test" and a procedure called "populate_sql_trace_test" which simply inserts some values to our table and runs a SELECT query. We trace the execution of this procedure.

We set the dump file specifications, then enable 10046 Event, that outputs the trace with wait event statistics. Then we located the dump file and name, and we used tkprof with some options to get tkprof output.

You can find the details of tkprof output and how to read and interpret the output. Also follow the topics on the blog, also I will post more topics about this subject.

Feed users, click here to go to this topic's page. You can find related topics and reference links and you can read the comments for this topic

Feed users, click here to post a comment for this topic

Note: There might be problems about HTML code and parsing and displaying HTML code of your viewer application. Also, there are problems caused by Google Reader on displaying of code samples on the entry. Also, the entry might be updated. So, it is recommended to visit the page of the entry.